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Preface 



Web engineering is a new discipline that addresses the pressing need for system- 
atic and tool-supported approaches for the development, maintenance and test- 
ing of Web applications. Web engineering builds upon well-known and success- 
ful software engineering principles and practices, adapting them to the special 
characteristics of Web applications. Even more relevant is the enrichment with 
methods and techniques stemming from related areas like hypertext authoring, 
human-computer interaction, content management, and usability engineering. 
The goal of the 4th International Conference on Web Engineering (ICWE 2004), 
in line with the previous ICWE conferences, was to work towards a better under- 
standing of the issues related to Web application development. Special attention 
was paid to emerging trends, technologies and future visions, to help the aca- 
demic and industrial communities identify the most challenging tasks for their 
research and projects. 

Following a number of successful workshops on Web engineering since 1997 
at well-known conferences, such as ICSE and WWW, the first conference on 
Web engineering was held in Caceres, Spain in 2001. It was followed by ICWE 
2002 in Santa Fe, Argentina and ICWE 2003 in Oviedo, Spain. In 2004 ICWE 
moved to the center of Europe and was held in Munich, Germany from July 26 
to 30. ICWE 2004 was organized by the Institute for Informatics of the Ludwig- 
Maximilians-Universitat (LMU) Munich. 

The ICWE 2004 edition received a total of 204 submissions, out of which 25 
papers were selected by the Program Committee as full papers (12% acceptance). 
Additionally, 60 papers describing ongoing research results were included, as ei- 
ther short papers or posters. The selected papers cover a wide spectrum of topics, 
including Web development processes, design methods, Web usability, security 
and performance, Web metrics, personalized and adaptive Web applications, the 
Semantic Web, and more. 

ICWE 2004 attracted people from five continents, with a wide and uniform 
geographical distribution of the papers’ authors. ICWE 2004 also hosted for 
the first time a tool demonstration track, and featured a two-day tutorial and 
workshop program, consisting of five tutorials and four workshops. Workshops 
and tutorials gave to the conference attendees the opportunity to enjoy a more 
informal forum for discussing the newest Web engineering topics, from Web 
quality, to the development of secure Web applications, to MDA applied to the 
construction of Web applications. Links to the workshops and tutorials can be 
found at the conference Web site: www.icwe2004.org 

We wish to express our gratitude to all the individuals and institutions who 
made this conference possible. We are very grateful to Lutz Heuser, Gerti Kappel 
and Roel Wieringa for presenting their keynotes at the conference. We would like 
to acknowledge all workshop organizers and tutorial presenters and the local or- 
ganizing committee. Our thank goes also to the workshop and tutorial co-clrairs 
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as well as to the demo and poster chair for their engagement. In particular, 
Maristella Matera did an excellent job managing the workshops organization. 
Our special thanks go to the program committee members and additional ref- 
erees for the very professional work done during the review process. The Online 
Conference System (OCS) was used during the review process for bidding and 
gathering submitted papers and reviews. We would like to thank the technical 
support team (Martin Karusseit and Markus Bajohr) at the University of Dort- 
mund, which provided invaluable assistance on the use of the Online Conference 
Service of METAFrame Technologies. We are also grateful to Springe r-Verlag for 
their helpful collaboration and quick publication schedule. Our deep recognition 
is due to Florian Hacklinger for his contribution in setting up and updating the 
conference Web site. 

Last but not least, we want to thank the more than 450 authors from over 
35 countries who contributed to this book and the conference. We thank them 
for their submissions and presentations and for providing us with the material 
in time. We count on them for the next conference edition, ICWE 2005, to be 
held in Sydney, Australia. 

Munich and Milan, May 2004 Nora Koch 

Piero Fraternali 
Martin Wirsing 
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The Real World or Web Engineering? 



Lutz Heuser 

Vice President Global Research & Innovation 
SAP AG 

A couple of years ago practitioners and researchers began to realize that the 
traditional issues covered in software engineering do not fulfil all of the requirements 
raised by the proliferation and growth of web-based information and application 
systems. 

“Fundamental differences [between hypermedia and other disciplines] 
however, make a pure transposition of techniques both difficult and 
inadequate. An important part of hypertext design concerns aesthetic 
and cognitive aspects that software engineering environments do not 
support,” 

[Nanard and Nanard, 1995] 

As a result, researchers currently make significant and ongoing contributions to a 
discipline that is called Web Engineering (WebE). Web Engineering aims to address 
and resolve the multifaceted problems of Web-based systems development. Even 
there are quite some different definitions for Web Engineering, the main focus of is 
usually on the establishment and use of sound scientific, engineering, and 
management principles and systematic approaches to the successful development, 
deployment and maintenance of high-quality Web-based systems and applications 
[San Murugesan, Yogesh Deshpande: Workshop Web Engineering ICSE2000]. 

Web Engineering is the application of systematic, disciplined and 
quantifiable approaches to the cost-effective development and evolution 
of high-quality applications in the World Wide Web. 

[www.webengineering.org, March 2004] 

“Web Engineering is a discipline among disciplines, cutting across 
computer science, information systems, and software engineering, as 
well as benefiting from several non-IT specializations” 

“While Web Engineering adopts and encompasses many software 
engineering principles, it incorporates many new approaches, 
methodologies, tools, techniques, and guidelines to meet the unique 
requirements of Web-based systems. Developing Web-based systems is 
significantly different from traditional software development and poses 
many additional challenges” 

[IEEE Multimedia] 

After taking a closer look at the Web Engineering principles and approaches that 
have already been developed, we recognise that most of these concepts assume that 
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systems and applications are developed from scratch and that the target environment 
is the smaller/mid-size business environment. 

Both of these assumptions do not apply to many of the established software 
companies and especially not to SAP. Founded in 1972, SAP is the recognized leader 
in providing collaborative business solutions for all types of industries and for every 
major market. SAP solutions are installed at more than 60,000 customer locations in 
120 countries. Comprehensive solutions are currently available for 23 distinct 
industries and each solution is tailored to the specific standards, processes, and 
challenges of the respective industry. And they’re developed, implemented, and 
supported by 28,700 professionals operating out of a global network of offices. 

Thanks to SAP’s unified technology platform, all solutions share common values, 
including: 

• Seamless integration: Removes the barriers that stand between people, systems, 
and information 

• Scalability: Accommodates virtually unlimited growth 

• Adaptability: Allows easy customization of features and functions - and helps to 
cope with constant change 

• Ease of implementation: Helps the customer to get up and running sooner 

• Lower total cost of ownership: Helps minimize long-term costs 

• Industry expertise: Supports the real-world processes employed every day 

As SAP has always been committed to the development of technology-independent 
business solutions, the evolution from the mainframe-based R/2 system to R/3 for 
Client/Server and Distributed Open Systems was just as natural as the progression 
towards web-based systems. 

Now SAP has announced that the future product portfolio will be based on an 
enterprise service architecture utilising web services on common business objects and 
business processes. This requires a "real world" definition of what a Web Service 
could be. 

SAP’s software engineering process is based on the Product Innovation Lifecycle 
(PIL), which describes how SAP invents, produces, and manages products throughout 
their entire lifetime from invention to transition and across multiple releases. The PIL 
is primarily driven by Portfolio & Solution Management and involves virtually all of 
SAP’s line of business functions (e.g. development, production, support). 




Fig. 1. SAP’s Product Innovation Lifecycle 
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The SAP Product Innovation Lifecycle (PIL) is based on a waterfall model that can 
be complemented by spiral/parallel approaches within PIL phases as needed. 

The deliverable-oriented phases of the PIL transform ideas into software releases, 
which in turn are deployed to create successful implementations. 



Ideas 



Invent 



Qualified 

Opportunities 



Define 



Development 

Contract 



Develop 



SAP validated 
Product 




Market validated 
Product 



Optimize 



Market accepted 
Product 



Fig. 2. Information flow in the PIL 



Having said this, the following sections will sketch some of the key challenges in 
the discipline of Web Engineering where SAP would like to evaluate how the 
concepts of numerous ambitious and skilful researchers could be implemented in the 
real world. The examples also outline how the scientific Web Engineering phases 
could be mapped to SAP’s PIL. 



1 Example: WebE: Requirement Engineering / PIL: Invent, 

Define 

Successfully defining the stakeholders' requirements is critical to the development of 
an application that delivers quality and meets expectations. Difficulties during the 
development process or creation of an unsuccessful application usually result from 
mistakes such as missing or not well-defined requirements as well as inclusion of 
superfluous requirements. 

In the real world, application development is often not carried out for a single 
customer but for a market segment. Standard applications have to fulfil the needs for a 
superset of customers of a specific business area (e.g. Enterprise Resource Planning, 
Human Resources) or coming from a specific industry (e.g. Automotive, Retail). 

Another real-world aspect that has to be addressed is the prioritisation of 
requirements according to the context of use with respect to the user as well as the 
application. Additionally, changes and advances in technology (e.g. user agents, 
standards, protocols) dynamically influence the set and prioritisation of requirements. 

Starting with a wide array of concepts and tools for the selection of relevant 
requirements, the Requirement Engineering phase is arguably lacking in well-defined 
and systematic support for the each of its components, which include the formal 
description, validation, management, and risk assessment of requirements as well as 
the cost forecasting/estimation analysis. 

Within SAP, it is agreed that any development to be initiated requires a so-called 
business case. It clearly defines why the development should be done in terms of 
customer demands or market opportunities. It is also necessary to justify required 
investments and to position the possible solution in relation to the competition. 
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2 Example: WebE: Analysis, Design / PIL: Define, Develop 

For the analysis and design aspects of Web Engineering, the initial steps in the PIL 
Development phase are the detailed planning and the detailed design. Keeping in 
mind that SAP products are used at 60,000 customer locations in more than 120 
countries, the design of the applications has to pay close attention to unique cultural 
aspects, localisation, and configuration of the local IT infrastructures. 

Web Engineering often arranges the analysis and design activities into orthogonal 
dimensions for content, interaction, navigation, presentation, processes, and 
communication. Flowever, the reality is that this orthogonal categorization doesn’t 
necessarily map well to existing systems and earlier development, as they are often 
stored in legacy systems, follow earlier process models, or have been set by the 
customer. 

In addition, current scientific approaches often do not fully take changing 
technologies and even changing development and architectural paradigms (e.g. 
client/server towards enterprise service architectures) into consideration when 
establishing analysis and design concepts for broad use. 

With respect to these aspects, the implementation of the Analysis and Design 
concepts developed in Web Engineering, have not been previously implemented very 
often because they do not take into account pre-existing legacy systems and 
development and sometimes neglect the reality that technology is constantly evolving. 



3 Example: WebE: Implementation / PIL: Develop 

In 1992, Derek Patridge stated, “Conventional software-system development is the 
process of designing a correct HOW for a well-defined WHAT.” Whereas the 
“WHAT” has already been defined in the previous phases of the Web Engineering 
process, the “HOW” should now clarify details about the implementation. 

As a result, emerging Web Engineering concepts often try to answer this by 
answering the question regarding “HOW the latest Web-technologies (e.g. SMIL, 
XForms, XSL-FO) could be applied.” Whenever a new version of a particular Web- 
technology appears, the answer it provides is usually a new and of course improved 
solution for the “HOW” compared to the previous version. 

In the real world, Partridge’s question about the “HOW” could meanwhile be 
rephrased as, “HOW all of these technologies should be combined and HOW to keep 
pace with the technological evolution?” Neither concepts that have been previously 
developed nor the tools that are currently being used allow developers and researchers 
to use the latest technology and to use them in conjunction with legacy technologies. 



4 Conclusion 

Beginning with the provoking question, “The real world or Web Engineering?” this 
brief overview has provided on one hand a short review of the roots of the discipline 
of Web Engineering and on the other hand introduced SAP’s background and Product 
Innovation Lifecycle. Three examples also demonstrated where a mismatch between 
SAP’s real world requirements and scientific Web Engineering results could be seen. 
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The conclusion however is that Web Engineering is a scientific discipline with a 
solid theoretical basis that delivers valuable and important pieces in solving the 
puzzle of how to build large, complex web-based systems. 

Research opportunities have been identified to answer questions such as, “How 
should these pieces be put together?” and “What is the best way to deal with the rapid 
evolution of technology and the resulting heterogeneity of these systems?” 
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Abstract. Modern Web applications are full-fledged, complex software sys- 
tems. Therefore, the development of Web applications requires a methodologi- 
cally sound engineering approach called Web Engineering. It is not clear, how- 
ever. to which extent existing solutions from relevant areas, most notably soft- 
ware engineering can be reused as such for the development of Web applica- 
tions and consequently, if Web Engineering is really a discipline on its own. 
This paper highlights the characteristics of Web application development as 
found in existing literature thus providing a prerequisite for analyzing the ap- 
propriateness of existing engineering solutions. The characteristics are catego- 
rized according to four dimensions, comprising the software product itself, its 
development, its use and evolution as a cross-cutting concern. 



1 Introduction 

The World Wide Web has a massive and permanent influence on our lives. Economy, 
industry, education, healthcare, public administration, entertainment - there is hardly a 
part in our daily life that has not been pervaded by the World Wide Web. The reason 
for this omnipresence lies especially in the very nature of the Web, which is shaped by 
the global and permanent availability and comfortable and uniform access to often 
widely distributed information producible by anyone in the form of Web pages 
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[2, 25]. Web applications today are full-fledged, complex software systems providing 
interactive, data intensive and customisable services accessible through different de- 
vices, working state-based for the realization of user transactions and usually storing 
the used data in an underlying database. Despite the fundamental changes in the ori- 
entation of the Web, from an informational to an application medium, the develop- 
ment of Web applications is still seen as a one-time event, spontaneous, usually based 
on the knowledge, experiences and development practices of individual developers, 
limited to be reused in the sense of the “Copy&Paste paradigm”, and ultimately char- 
acterized by inadequate documentation of design decisions. A survey of the practice of 
Web application development reported that there is only limited use of design tech- 
niques, documentations are produced seldom and testing procedures are most often 
not formalized [31]. Keeping this practice in mind, it is no surprise that a survey done 
by the Cutter Consortium [7] found out that the top problem areas of large-scale Web 
application projects were the failure to meet business needs (84%), project schedule 
delays (79%), budget overrun (63%), lack of required functionality (53%), and poor 
quality of deliverables (52%). The current situation of ad hoc 1 development of Web 
applications reminds us of the software development practices of the 1960s, before it 
was realized that the development of applications required more than programming 
expertise [8, 9, 28]. 

Now, the problems seem to be the same, so are the solutions the same, too? Is the 
notion of Web Engineering just a new application domain of software engineering 
reusing and slightly adapting already existing approaches in terms of methodologies, 
principles, standards and best practice guides, or is it really a discipline in its own 
calling for new solutions (cf., e.g., [13, 15, 16])? One of the prerequisites to answer 
this question is to clarify, how far Web application development is different to tradi- 
tional software development. Answering this question would allow to reason about 
the applicability and appropriateness of approaches already existing in relevant com- 
puting fields. 



2 Characteristics of Web Application Development 

Web applications are software systems based on technologies and standards of the 
World Wide Web Consortium (W3C). They provide Web-specific resources such as 
content and services through a user interface, the Web browser. With this definition 
of Web applications in mind, we try to explore the characteristics of Web application 
development, heavily relying on existing literature [1, 5, 11, 15, 16, 20-29, 32], We 
structure our discussion into four different dimensions, comprising the software 
product itself, its development, its use and evolution as a cross-cutting concern [ 14, 
19]. It has to be emphasized that we don’t claim that each of these characteristics is 
unique to Web application development never occurring when developing traditional, 
i.e., non-Web applications. Characteristics mentioned in existing literature to be 



1 Note that the term "ad hoc” is interpreted as “doing something in an unstructured way”, and 
not in its original sense of “focused at the problem at hand” (cf. [12]). 
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unique for Web development seem to be not that outstanding compared to certain 
traditional software development domains [15, 16]. This is not least due to the fact 
that for each characteristic the degree of difference depends on the category of Web 
application considered (examples for such categorizations can be found in [19, 26, 
29]). 

2.1 Application-Related Characteristics 

When developing Web applications one has to consider not only functionality but 
equally address content, hypertext, and presentation aspects. 

Content. The origin of the Web is its role as information medium. Beyond the re- 
quired functionality, Web applications are thus heavily content-driven. Content com- 
prises not only structured data residing in database systems but also unstructured and 
semi-structured data such as textual descriptions or multi-media information. Com- 
plexity arises especially from the fact that content is often highly dynamic and con- 
tinuously updated. Also, users typically demand high content quality in terms of topi- 
cality, accurateness, consistency, and reliability [30]. Consequently, the development 
of Web applications is not only a complex engineering task but relies heavily on 
authors responsible for the content. 

Hypertext. Web applications advocate the hypertext paradigm [6] as the funda- 
mental paradigm for structuring information. The basic elements of the Web’s notion 
of hypertext are nodes, links and anchors. Typical examples of accessing hypertextual 
information include browsing (like in online stores' catalogues), querying (like in e- 
learning applications), or guided tours (like in virtual exhibitions). The essential fea- 
ture of the hypertext paradigm is its non-linearity requiring from both authors and 
users to address the potential issues of disorientation and cognitive overload. This can 
be achieved for example through specific navigation design (site maps, keyword 
searches, traversed paths, etc.) and is essential to preserve quality of access [10]. 

Presentation. In conventional software systems the "look and feel" is often to a 
large extent determined by standardized user interface elements and style guides. 
Presentation is a central quality factor for Web applications not least to the high com- 
petitive pressure on the Web where visual appearance is subject to (ever-changing) 
fashion, trends, and new technical features [29] . In addition, as application designers 
cannot expect Web users to consult a user’s manual Web applications need to be self- 
explanatory requiring particular attention to visual design and the consistency of the 
interaction style behaviour. 



2.2 Usage-Related Characteristics 

Unlike in more traditional settings, the users of Web applications often vary in num- 
bers and cultural background, use heterogeneous devices and can freely choose the 
time and location of accessing the Web application [18]. Developers frequently can- 
not predict all these potential settings. 
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Natural context. This includes aspects of the location and time of access, offering 
the opportunity of new kinds of context-based services, not least due to the advent of 
mobile computing. In addition, the possibility of immediate and permanent availabil- 
ity of Web applications requires special quality considerations such as 24/7 availabil- 
ity. 

Unpredictable technical infrastructure. Available end-user devices vary in 
hardware and software capabilities such as display size, computational power, or 
browser version. Also network connections differ with respect to bandwidth, reliabil- 
ity, stability and availability, all affecting the quality of sendee [29]. Complexity is 
increased even further due to the fact that the actual representation of the Web appli- 
cation on the client device is to a large extent outside the control of the developers. 
For example, users configure their browsers individually and may even disallow 
certain essential features (e.g., cookies or JavaScript). 

Diversity and magnitude of user base. Web application users differ in age, social 
and cultural backgrounds, goals, intentions, skills, and capabilities [17]. These het- 
erogeneity has to be considered by application developers since the Web entails no 
obligation and Web applications will only be used if they bring immediate advantage. 
The way users interact with the Web application can be hardly predicted and users 
may leave the Web application at any time [15]. Also, the number of users accessing 
the Web application may vary considerably making scalability another crucial quality 
aspect. 



2.3 Development-Related Characteristics 

Web application developers need to deal with conditions, risks, and uncertainties not 
always present in traditional software projects. 

Development team. Web application development is a multi-disciplinary effort 
comprising a mixture of print publishers, authors, software developers, marketing 
experts, and art designers [26]. Such teams are also dominated by significantly 
younger team members which are less willing to adhere to conventions and more 
inclined towards applying new (and often still immature) technologies [24]. Another 
important characteristic is the involvement of open source communities. 

Development environment. The technical infrastructure used for developing a 
Web application is characterized by a high degree of volatility and heterogeneity. 
Web application development relies on a broad spectrum of different COTS compo- 
nents (e.g., Web server, application server, database system, publishing framework 
etc.). Because of the increased time-to-market pressure these components are often 
immature and fall short in stability, reliability, and desired functionality. 

Legacy integration. Web applications often need to integrate legacy systems [21 ]. 
The external services provided by these systems are, however, rarely documented and 
often change without notice, thus negatively affecting the quality of the overall Web 
application. 

Process. Web application development processes are characterized by frequent 
changes and adjustments, which are necessary due to rapid technological develop- 
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ments, fast changing trends, volatile requirements, and rigid schedules. This calls for 
highly iterative, flexible, and prototype-oriented development methods [3, 24]. 



2.4 Evolution-Related Characteristics 

Web applications are subject to frequent changes and permanent evolution. Their 
development is driven by rapidly changing technology and the volatility of Web users 
leads to a highly competitive situation where immediate Web presence and short time 
to market are considered crucial: "Unlike conventional application software that 
evolves over a series of planned, chronologically spaced releases, Web applications 
evolve continuously." [28], In the course of evolution, negotiability of quality often 
sacrifices maintainability [14, 20]. 



3 Conclusion 

To say it with the words of Robert Glass [ 13], “... there have always been important 
differences between software projects, diversity is the key part of software develop- 
ment. Information systems have always been developed differently from scientific 
systems, and critical projects have been treated differently from more mundane ones. 
Why should we be surprised that the same condition holds for Web and non-Web 
projects?” There are, however, important differences between the two, urgently re- 
quiring to systematically re-think the applicability and appropriateness of existing 
solutions in the whole area of software engineering, eventually based on the knowl- 
edge areas defined by the SWEBOK [4], The clarification of this issue would be a 
major step towards establishing Web Engineering as a discipline on its own. 
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1 Introduction 

Taken literally, the term “requirements engineering” (RE) is a misnomer. A 
requirement is something that is wanted; engineering, according to Webster’s, is 
calculated manipulation. If our wants would arise by calculated manipulation, 
then something would be wrong. Our wants should not be engineered. What 
should be engineered, are solutions that meet our wants. 

So what is requirements engineering? In this talk I discuss two views of RE, 
as problem analysis and as solution specification, and show that these two views 
meet in the discipline of IT systems architecting. 

Architecture is central to web engineering because the web is an infrastruc- 
ture for distributed coordination. Requirements engineering for web aplications 
therefore must deal with a distributed and sometimes fuzy set of stakeholders 
and with evolving requirements that will change once people use the application 
and explore new coordination mechanisms. In this context, requirements engi- 
neering is a distributed and concurrent process of problem analysis and solution 
specification. 

2 Requirements Engineering and Problem Solving 

The frequently heard mantra of software engineers is that requirements specify 
what a system should do, whereas a design says how it should do it. But the 
distinction between “what” and “how” is meaningless. We can ask how a system 
behaves externally and what its internal structure is, just as we can ask what its 
external behavior is and how it is structured internally. A better distinction is 
that between problem and solution. A problem is a difference between what is 
perceived to be the case and what is desired, that we want to reduce; a solution 
is an action that reduces this difference [1,2]. One view of requirements is that 
they specify a problem; another view is that they specify a solution. I discuss 
both views in this talk. 

Note that problem analysis and solution specification need not be sequentially 
related. In general, problem analysis and solution specification proceed jointly, 
with the engineer spending most, but not all of her time on problem analysis at 
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the start of the process, and spending most, but not all of her time on solution 
specificatioon towards the end of the process [3, 4] . 



3 Requirements Engineering as Problem Analysis 

If requirements specify a problem, then a requirements specification should de- 
scribe 

— what the problematic phenomena are, 

— what the cause relationships between these phenomena are, 

— by which norms these phenomena are problematic, and 

— which stakeholders have these norms. 

An example of this approach to RE is goal-oriented RE, which starts from a top- 
down analysis of user goals and refines these until desired properties of solutions 
are found [5-8]. Another example is the problem frame approach of Michael 
Jackson, in which frequently occurring problem structures are identified and are 
related to a frame concern, which relates the solution specification to a goal 
in the problem domain. A third approach sees problem-oriented RE as theory- 
building, in which a theory of the problematic phenomena is built, that will help 
us to specify a solution that takes away the causes of the problems [9]. All these 
approaches take a rather top-down approach in which the assumption is that 
users can specify their goals (even if these may be mutually inconsistent). In cases 
where users cannot specify their goals, yet other approaches are suitable, such as 
the task-support style of requirements specification proposed by Lauesen [10] or 
an evolutionary approach in which users are observed in their work, from which 
conclusions about their requirements are drawn [11,12]. 



4 Requirements Engineering as Solution Specification 

The view RE as solution specification is taken by the IEEE 830 standard [13] 
and by other authors on requirements [14, 15]. In this view, a requirements spec- 
ification consists of 

— a specification of the context in which the system will operate, 

— a list of desired system functions of the system, 

— a definition of the semantics of these functions, 

— and a list of of quality attributes of those functions. 

Classical techniques for software solution specification are structured analy- 
sis and, to some extent, object-oriented techniques [16, 17]. Note however that 
object-oriented techniques such as the UML are primarily oriented towards spec- 
ifying the internal structure of software solutions, not towards specifying their 
requirements. 
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5 IT Systems Architecture 

The two views of requirements come together in the concept of system architec- 
ture. I define an architecture as the set of relationships between the components 
of a system, that jointly ensure emergent properties of the system as a whole. 
The architecture of a building is the set of relationships between parts of the 
building that cause the building to have desired properties, such as room, shelter, 
functionality and appearance. In the same way, the architecture of a software 
system is the set of relationships between its components that cause the system 
to have desired properties, such as a desired functionality, behavior, semantics, 
and quality of service. 

Architecture links requirements as problem characteristics to requirements as 
solution properties. Consider a set of problem-oriented requirements, that char- 
acterize a business problem, and a set of solution-oriented requirements, that 
specify desired solution properties. If part of this solution is a software system, 
then other parts may be novel work procedures and a new physical layout of 
an office. Thus, the solution has an architecture that is expected to solve the 
business problem and the desired software system plays a role in this architec- 
ture. And it is the architecture that links the software solution to the business 
problem. The same holds, mutatis mutandis, at lower levels of the aggregation 
hierarchy. For example, the component architecture of a software system links 
the component specifications to the software problem that they are intended to 
solve. 

Architecture is the central problem in web applications because these appli- 
cations should enable distributed coordination between people, and the architec- 
ture of these coordination mechanisms evolves by itself as well as is designed by 
people. As far as it evolves by itself, it may link unforeseen solution properties to 
unforeseen problems. If left to its own, evolutionary processes tend to deteriorate 
the architecture of a system. The challenge of requirements engineering for web 
applications is therefore to design architectures that enable this process. 
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Abstract. A novel approach is presented for automatically evaluating of the us- 
ability and accessibility (U&A) of web sites by performing a static analysis of 
their HTML code against U&A guidelines. The approach relies on separating 
guidelines evaluation logic from the evaluation engine. Due to this separation, 
the whole evaluation process can be divided into two main phases: specifying 
formal guidelines and web page evaluation. In the first phase, the formal struc- 
ture of a guideline is expressed in terms of Guideline Definition Language 
(GDL). In the second phase, the web page is parsed to identify its contents and 
structure and link them to relevant guidelines to be evaluated on the page 
parsed. This approach enables the simultaneous evaluation of multiple guide- 
lines selected on demand from different sources. It also optimises evaluation by 
automatically identifying common sub-structures among structured guidelines. 
It also supports the expression, by evaluators with different usability practises, 
of alternative evaluation strategies. 



1 Introduction 

The World Wide Web has become a predominant mean for communicating and pre- 
senting information on a broad scale and to a wide audience. Unfortunately, web site 
usability and accessibility continue to be a pressing problem [1], An estimated 90% of 
sites provide inadequate usability [2], and an estimated 66% of sites are inaccessible 
to users with disabilities [3]. Although numerous assistive devices, such as screen 
readers and special keyboards, facilitate use of web sites, these devices may not im- 
prove a user’ s ability to find information, purchase products and complete other tasks 
on sites. For example, sites may not have links to help blind users skip over naviga- 
tion bars, or sites may not enable users to increase the text font size, so that they can 
read it. A wide range of Usability and Accessibility (U&A) evaluation techniques 
have been proposed and a subset of these techniques is currently in common use. 
Automation of these techniques became much desired [4,5] because they required 
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U&A specialists to conduct them or to analyse evaluation results, which is very re- 
source consuming especially for very large, continuously growing web sites. In addi- 
tion, there is a lack of experts due to an increased demand. A possible solution to that 
problem consists in relying on U&A guidelines and recommendations to be reviewed 
and applied by designers and developers. Some studies show that applying guidelines 
by designers is subject to interpretation, basically because of the inappropriate struc- 
turing or formulation [6], For this reason and others, automation has been predomi- 
nately used to objectively check guideline conformance or review [5], Several auto- 
matic evaluation tools were developed to assist evaluators with guidelines review by 
automatically detecting and reporting ergonomic deviations from these guidelines and 
making suggestions for repairing them. In this paper, a novel approach is presented 
that automate the evaluation of a web site against U&A guidelines by checking a 
formal representation of these guidelines on the web pages of interest. The aim of the 
approach is to overcome the major shortcomings of existing tools. The main charac- 
teristic of this approach is the separation between the evaluation logic and the evalua- 
tion engine. In this way, the U&A guidelines can be expressed in terms of conditions 
to be satisfied on HTML elements (i.e., tags, attributes). A formal specification lan- 
guage supporting this approach implements a framework [7] that enables the trans- 
formation of such U&A guidelines from their initial expression in natural language 
into testable conditions on the HTML code. Once expressed, the guidelines can be 
evaluated at evaluation-time by configuring their formal expression in an improved 
way depending on the guidelines to be evaluated and the HTML elements contained 
in the page. This process consequently considers guidelines relevant to the targeted 
evaluation context, and factors out sub-structures that are common across these 
guidelines, even if they come from different sets of guidelines. 

This paper is structured as follows: Section 2 briefly describes some automatic 
U&A evaluation tools. Section 3 presents a global view of the evaluation process and 
the fundamental concepts of our evaluation approach. Section 4 exemplifies the ap- 
proach on guidelines which is not found in existing tools. Section 5 underlines the 
possibilities of evaluation improvement, the most original part of our approach. Sec- 
tion 6 concludes the paper by stressing major advantages of the proposed approach. 



2 Related Work 

Depending on the evaluation method, U&A may involve several activities such as: 

1. Capture: it consists of collecting U&A data, such as task completion time, errors, 
guideline violations, and subjective ratings. 

2. Analysis: it is the phase where U&A data are interpreted to identify U&A prob- 
lems in the web site. 

3. Critique: it consists of suggesting solutions or improvements to mitigate the pre- 
viously identified problems. 

Many evaluation tools were developed to provide automation of some of the above 
activities. In this section we would like to report on some tools used for Web evalua- 
tion by guideline review, a particular evaluation method that has been selected for its 
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simplicity, its capability to be conducted with or without users, and its wide applica- 
bility. Some of the tools are dedicated only to usability, some others to accessibility, 
but none of them to both U&A: 

• A-Prompt [9] is an off-line tool designed to improve the usability of HTML 
documents by evaluating web pages for accessibility barriers and then providing 
developers with a fast and easy way to make the necessary repairs. The tool's 
evaluation and repair checklist is based on accessibility guidelines created and 
maintained by the Web Accessibility Initiative (http://www.w3.org/WAI). 

• Bobby [10] is a comprehensive web accessibility tool designed to help evaluate 
and repair accessibility errors. It tests for compliance with WCAG1.0 and Section 
508 guidelines. It offers prioritised suggestions based on the WCAG1.0. It also 
allows developers to test web pages and generate summary reports highlighting 
critical accessibility issues sorted by rank on priority. 

• LIFT OnLine (http://www.usablenet.com) tests web pages against a subset of all 
usability and accessibility guidelines (rules), and then sends an e-mail with the 
link to a usability report on line. As soon as the analysis is completed, LIFT On- 
line shows a list of the pages in the web site containing potential usability prob- 
lems. Each problem is ranked by severity and is described in details. 

• WebSAT (http://zing.ncsl.nist.gov/WebTools/WebSAT/overview.html) inspects 
the HTML composition of web pages for potential usability problems. It can per- 
form inspections using either its own set of usability rules or those of the IEEE 
Std 2001-1999. 

• WebCriteria (http://www.webcriteria.com) is a tool for comparative evaluation of 
a web site with respect to a benchmark derived from similar well-established web 
sites that are considered as reference. 

The common major shortcoming of the above existing U&A evaluation tools is that 
the evaluation logic is hard coded and hard wired in the evaluation engine, which 
makes them very inflexible for any modification of the evaluation logic. Introducing a 
new guideline, possibly a custom one, or modifying an existing guideline remains 
impossible. In addition, many of them do not offer many possibilities of controlling 
the evaluation process like choosing which guideline to evaluate, the level of evalua- 
tion at evaluation time, or the level of priority. For example, Bobby only provides the 
choice of the guidelines set to evaluate: W3C or Section508. 



3 The Evaluation Approach 

To address the above shortcomings, the evaluation process is structured in our ap- 
proach by decomposing the whole evaluation process into two distinct but related 
phases: specifying formal guidelines which is achieved only once before any evalua- 
tion and evaluating a web site, which is conducted at any time. The two main phases 
remain totally autonomous, thus giving many possibilities to improve each of them. 
The different steps of these phases are now further described. 
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Structuring. The first step in our approach consists of structuring U&A guidelines in 
terms of evaluation conditions so as to obtain a formal guideline, expressed in a for- 
mat that can be processed by an automaton as opposed to the natural language format 
of the initial guideline found in the literature. Guidelines structuring requires a thor- 
ough understanding of both the U&A knowledge and the HTML language to bridge 
the gap between them. It is also influenced by the understanding of the original 
guideline semantics that may lead to several interpretations of the same guideline. 
The formal guideline is expressed according to Guideline Definition Language 
(GDL), a XML-compliant language supporting the approach by formalising the 
structuring of U&A guidelines toward automatic evaluation. The most original char- 
acteristic of the GDL is its naturalness, i.e. the possibility offered by the language to 
straightforwardly map the informal statement of initial guidelines onto formal state- 
ment expressed in the GDL language. GDL is aimed at modelling HTML evaluable 
aspects only (e.g., colour contrast, alternative text for visual content). Other frame- 
works have to be used to cope with other direct usability aspects such as user satis- 
faction, consistency, and information organisation. Meta-information is then added to 
each GDL-compliant guideline according to taxonomy on indexes [?]. 




Fig. 1 . The two main phases of evaluation process based on the proposed approach 



Site crawling. The web pages of interest, whether they are static or dynamically gen- 
erated, are simply specified by their URL, automatically downloaded from the web 
and stored in a dedicated directory where the rest of the process will take place. 

Page parsing. This step is done by a single scan for each web page to be evaluated. 
During this scan, the parser captures the instances of the evaluation sets specified in 
the formal guidelines to be checked. Parsing parameters can be specified to control 
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this process: which guidelines to evaluate, number of desired instances (e.g., a given 
maximum, all instances), whether incomplete set instances could be kept, etc. 
Evaluation. After parsing the web page, the evaluation conditions that have been de- 
fined during guidelines structuring in GDL are processed to check their satisfaction 
by performing a property checking. Every condition is applied on the captured in- 
stances of its corresponding set to determine its respect or violation. In this way, a 
detailed evaluation report can be generated on respected/violated sets, number of de- 
tected instances, and percentage of respect/violation. 



4 Fundamental Concepts 

To facilitate the different phases of the evaluation process, several new concepts had 
to be identified and defined. Fig. 2 depicts a global view of the fundamental concepts 
of our approach and their interactions. These concepts form the building blocks of 
GDL. To exemplify how GDL can be exploited to transform a natural language 
guideline into a formal counterpart, let us select a first guideline: “Use a limited num- 
ber of fonts in a single web page”. 




Fig. 2. Fundamental Concepts 



As guidelines are generally expressed at a high level of abstraction, the original 
guideline expressed in natural language should be transformed into a concrete form 
that can be understood by designers and manipulated by the system. This re- 
expression is called an interpretation of the guideline. In general, interpretation is 
used to limit the focus of the original guideline to some extent that can be considered 
satisfactory in the targeted evaluation context. Of course, even with interpretation, 
evaluation of some guidelines cannot be totally automated. For this reason, every in- 
terpretation is assigned to a factor indicating the level of abstraction reflected. In our 
example, the guideline needs to be interpreted because "limited number" is very ab- 
stract. If we have an evaluation logic that can be applied on many evaluation sets with 
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some difference, we can define a meta evaluation condition for all these sets, then we de- 
fined a mapped evaluation condition for every evaluation set. Otherwise, we specify a di- 
rect evaluation condition. In a meta evaluation condition, we use meta variables to specify 
the evaluation logic. The instantiation of the condition for a given evaluation set is done 
by mapping between the Meta variables and the corresponding concrete set elements. 
Global conditions (Meta or direct) are formed of operations. These operations provide a 
mechanism for identifying potential common parts among evaluation conditions. They are 
assigned a priority indicator to avoid ambiguity and to facilitate the execution of global 
conditions. A direct condition that can be associated to the evaluation set of our example 
could be NBRJNSTANCES(S1)<4, where NBRJNSTANCES is a predefined GDL func- 
tion. In the GDL expression below corresponding to the condition, we start by finding the 
number of set instances captured from the parsed web page, then we test if it is smaller 
than 4. 

<Direct_Condition set=”S1”> 

<Operation id=”01” 

Symbol=”NBRJNSTANCES” 
return=”number” Order=”1”> 

<Arg type=”Set” value=”S1”/> 

</Operation> 

<Operation id=”02” 

Symbol=”<” retum=”Bool” Order=”2”> 

<Arg type=”Operation” value=”01”/> 

<Arg type=”number” value=”4”/> 

</Operation> 

</Direct_Condition> 



5 Examples with Evaluation Improvement 

5.1 Basic Examples with Meta Condition 

The evaluation approach is now applied on two reasonably complex guidelines. The 
first one is: GDLl=“Select colors that will make your page easy to read by people with 
color blindness”[10]. As such, GDL1 cannot be automated in a straightforward manner 
as there is no calculable way to assess to what extent a page is easy to read or not, de- 
pending on the users. However, if we refer to the research conducted by Murch [11], 
an interpretation lnter_GDL1 of this guideline can be produced: “The combination be- 
tween background color and foreground color should belong to the best color combina- 
tions or should not belong to the worst color combinations proposed by Murch”. This in- 
terpretation will not cover all colors because Murch dealt with basic color only, but we 
will use it for simplification. For lnter_GDL1, we have the following evaluation sets: 

• Slcontrols text color over the whole page. Sl={Body.bgcolor Pase , Body. text Pa8e }. 
<SET id=”S1” name=”Global color control” priority=”AAA”> 

<Element id=”E1” tag="Body” Attribute=”text” scope=”Page”/> 

<Element id=”E2” tag="Body” Attribute=”bgcolor” scope=”Page”/> 

</SET> 

• S2 controls color by Body and Font. S2={Body.bgcolor Pas \ Font.color Body bgcolor }. 

<SET id=”S2” name=”Body Font color control” priority=”AAA”> 

<Element id=”E2” tag="Body” Attribute=”bgcolor” scope=”Page”/> 
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<Element id=”E3” taq=”Font" Attribute=”color” scope=”E2”/> 
</SET> 



The remaining evaluation sets can be obtained by analogy: 

•• S3 controls color in Font and Table. S3={Table.bgcolor Page , Font.color Tableb8COl ° r }. 

• "S4: controls color in Font and TH. S4={TH.bgcolor Page , Font.color THbgcol ° r }. 

••• S5: controls color in Font and TR. S5={TR.bgcolor Page , Font.color TRbgcol ° r }. 

••• S6: controls color in Font and TD. S6={TD.bgcolor Page , Font.colorTD. b8COlor }. 

••• S7: controls color in Body and TH. S7={TH.bgcolor B ° dy, “', Body.text Page }. 

•• S8: controls color in Body and TR. S8={TR.bgcolor Bo<iytext , Body.text Page }. 

•• S9: controls color in Body and TD. S9={TD.bgcolor Bodytext , Body.text Page }. 

•• S10: controls color in Body and Table. S10={Table.bgcolor Bo<iytext , Body.text Page }. 

According to our experience with HTML 4.0, these sets cover all the possibilities pro- 
vided by HTML to manipulate color of normal text (not links). So, we can consider 
that the evaluation of lnter„GDL1 (and thus GDL1) can be totally automated. The 
evaluation conditions associated with the above sets are very similar. Thus, we can de- 
fine a Meta evaluation condition that can instantiated for every evaluation set by map- 
ping the Meta variables to corresponding concrete set elements. The Meta evaluation 
condition corresponds to the next pseudo specification (PS): 

(BackgroundColor IN ListOfMurchColors) AND 
(ForegroundColor IN ListOfGoodColors(BackgroundColor)) OR 
(ForegroundColor NOT IN ListOfBadColors(BackgroundColor)) 



where ListOfGoodColors and ListOfBadColors are two lists of predefined values (colors). 
As mentioned earlier in this section, we will use Murch color combinations. For this 
purpose, basic colors are defined as user values: 



<User_V id=”Black” 
<User_V id=”White” 
<User_V id=”Red” 
<User„V id=”Geen” 
<User_V id=”Blue” 
<User_V id=”Cyan” 
<User_V id=”Magenta” 
<User_V id=”Yellow” 



type=”Color” val=”#000000”/> 
type=”Color” val =”#ffff ff”/> 
type=”Color” val=”#ff0000”/> 
type=”Color” val=”#00ff00”/> 
type=”Color” val=”#0000ff”/> 
type=”Color” val=”#00ffff”/> 
type=”Color” val=”#ff00ff”/> 
type=”Color” val=”#ffff00”/> 



Then, we specify lists of values corresponding to Murch colors, good and bad fore- 
ground colors for a given background color: 

<User„V id=”MurchColors” type=”Sequence” 

val=”Black White Red Green Blue Cyan Magenta Yellow”/> 

<User__V id=”GoodFgBlackBg” type=”Seq” val=”White Yellow”/> 

<User_V id=”BadFgBlackBg” 

type=”Sequence” val=”Blue Red Magenta”/> 

<User_ V id=”GoodFgWhiteBg” 

type=”Sequence” val=”Blue BlackRed”/> 

<User_V id=”BadFgWhiteBg” 

type=”Sequence” val=”Yellow Cyan”/> 

<User_V id=?GoodFgRedBg? 

type=”Sequence” val=”White Yellow”/> 

<User_V id=”BadFgRedBg” 

type=”Sequence” val=?Magenta Blue”/> 
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<User„V id=”GoodFgGreenBg” 

type=”Sequence” val=”Black Blue Red”/> 

<User_V id=”BadFgGreenBg” 

type=”Sequence” val=”Magenta Cyan ”/> 

<User_V id=”GoodFgBlueBg” 

type=”Sequence” val=”Green Red”/> 

<User_V id=”BadFgBlueBg” 

type=”Sequence” val=”Red Magenta”/> 

<User_V id=”GoodFgCyanBg” 

type=”Sequence” val=”White Yellow”/> 

<User_V id=”BadFgCyanBg” 

type=”Sequence” val=”Yellow White”/> 

<User„V id=”GoodFgMagentaBg” 

type=”Sequence” val=”White Black”/> 

<User_V id=”BadFgMagentaBg” 

type=”Sequence” val=”Green Red”/> 

<User„V id=”GoodFgYellowBg” 

type=”Sequence” val=”Black Blue”/> 

<User_V id=”BadFgYellowBg” 

type=”Sequence” val=”White Cyan”/> 

Then, a meta condition can be defined corresponding to the above pseudo specifica- 
tion (PS). One will easily notice that specification of conditions is relatively long. This 
is due to our desire to provide a GDL specification language that is as flexible and 
rich as possible, 

<Meta_Condition MCJD="MurchModel"> 

<Meta_Vars> 

<Meta_Var Name="BgColor" Type="Color"/> 

<Meta„Var Name="FgColor" Type="Color"/> 

</Meta_Vars> 

<Model Expression=””> 

<Operation id=”OT Symbol=”IN” return=”Bool” Order=”1” 

Stop_Val=”False” Stop_Msg=”Unrecognized Background color.”> 

<Arg type=”Vaf’ value^’BgColof’ Pos=”1”> 

<Arg type=”value” value=”MurchColors” Pos=”2”> 

</Operation> 

<Operation id=”02” Symbol=”IN” return=”Bool” Order=”2” 

Stop„Val=”True” Stop_Msg=”Good color combination. ”> 

<Arg type^’Vaf’ value=”FgColor” Pos=”1”> 

<Arg type=”value” value=”GoodFg(BgColor)” Pos=”2”> 

</Operation> 

<Operation id=”03” Symbol=”NOT IN” return=”Bool” Order=”3” 

Stop„Val=”True” Stop_Msg=”Not bad color combination. ”> 

<Arg type^’Vaf’ value=”FgColor” Pos=”1”> 

<Arg type=”value” value=”BadFg(BgColor)” Pos=”2”> 

</Operation> 

<Operation id=”04” Symbol=”OR” return=”Bool” Order=”3” 

Stop_Val=”True” StopJVIsg=”Good color combination.” Stop_Val=”False”> 

<Arg type=”Op” value=”02” Pos=”1”> 

<Arg type=”Op” value=”03” Pos=”2”> 

</Operation> 

<Operation id=”05” Symbol=”AND” return=”Bool” priority=”3”> 

<Arg type=”Op” value=”01”> 

<Arg type=”Op” value=”04”> 

<Operation> 

</Model> 

</Meta_Condition> 
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Notice that the above specification is long because we wanted to have complete con- 
trol over the execution by using the concept of Stop Value that allows stopping the exe- 
cution after any operation if its result corresponds to a given value. In this way, we can 
generate highly customised output messages via the Stop Msg. The evaluation expres- 
sion can also be specified as a single piece of text (the Expression attribute of the Model 
element) to be interpreted by the evaluation engine at execution time. This way is 
clearer and shorter than the above way, but the specified expression must respect a 
predefined syntax to be correctly interpreted. After defining the meta condition we in- 
stantiate it on every evaluation set via the mapping rules. 

<Mapped_Condition SetJD="S1" Meta_ID="MurchModel"> 

<MetaJVIapping Meta="BgColor" lnstance="E1"/> 

<MetaJVIapping Meta="FgColor" lnstance="E2"/> 

</Mapped_Condition> 

<Mapped_Condition Set_ID="S2" MetaJD="MurchModel"> 

<MetaJVIapping Meta="BgColor" lnstance="E1"/> 

<Meta_Mapping Meta="FgColor" lnstance="E3"/> 

</Mapped_Condition> 

Other mappings can be defined similarly for the remaining sets. Let us know consider 
another guideline oriented toward accessibility: “Web pages shall be designed so that 
all information conveyed with color is also available without color” (Section 508). 
GDL2 needs to be interpreted as well. As the guideline suggests, information con- 
veyed by color can be conveyed using markup. We will consider the following 
markup tags: Bold <b>, Italic <i>, text size <Font.size> and text font <Font.face>. Thus, 
the interpretation of GDL2 could become Inter. GDL2: “Web pages shall be designed so 
that all information conveyed with color is also available using any combination of the 
above markup elements”. This means that, in our evaluation context, lnter_GDL2 is 
considered violated even if colored information was conveyed using other means than 
the above markup tags. For lnter_GDL2, we have the following evaluation sets: 

• SI A: conveying colored information using bold tag. 

<SET id=”S1A” name=”bold conveyance” Priority=”AAA”> 

<Element id=”E1” tag=”Body” Attribute=”bgcolor” scope=”Page”/> 

<Element id=”E2” tag=”Font” Attribute=”color” scope=”E1”/> 

<Element id=”E3” tag=”b” Attribute^’” scope=”E2”/> 

</SET> 

• SIB: conveying colored information using bold tag. 

<SET id=”S1B” name=”bold conveyance” Priority=”AAA”> 

<Element id=”E1” tag=”Body” Attribute=”bgcolor” scope=”Page”/> 

<Element id=”E3” tag=”b” Attribute^’” scope=”E1”/> 

<Element id=”E2” tag=”Font” Attribute=”color” scope=”E3”/> 

</SET> 

• S2A: conveying colored information using italic tag. 

<SET id=”S2A” name=”italic conveyance” Priority=”AAA”> 

<Element id=”E1” tag=”Body” Attribute=”bgcolor” scope=”Page”/> 

<Element id=”E2” tag=”Font” Attribute=”color” scope=”E1”/> 

<Element id=”E4” tag=”i” Attribute^’” scope=”E2”/> 

</SET> 

• S2B: conveying colored information using italic tag. 

<SET id=”S2B” name=”italic conveyance” Priority=”AAA”> 

<Element id=”E1” tag=”Body” Attribute=”bgcolor” scope=”Page”/> 

<Element id=”E4” tag=”i” Attribute^’” scope=”E1”/> 
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<Element id=”E2” taq=”Font” Attribute=”color” scope=”E4”/> 

</SET> 

• S3: conveying colored information using font face. 

<SET id=”S3” name=”font face conveyance” Priority=”AAA"> 

<Element id=”E1” tag=”Body” Attribute=”bgcolor” scope=”Page”/> 

<Element id=”E2” tag=”Font” Attribute=”color” scope=”E1”/> 

<Element id=”E5” tag=”Font” Attribute=”face” scope=”E2”/> 

</SET> 

• S4: conveying colored information using font size. 

<SET id=”S4” name=”font size conveyance” Priority=”AAA"> 

<Element id=”E1” tag=”Body” Attribute=”bgcolor” scope=”Page”/> 

<Element id=”E2” tag=”Font” Attribute=”color” scope=”E1”/> 

<Element id=”E6” tag=”Font” Attribute=”size” scope=”E2”/> 

</SET> 

Notice that we defined S1A and SIB to cover the case of bold tag. We needed to do 
so because the two expressions <b><Font color=‘‘...‘‘>Colored bold text</Fontx/b> and <Font 
color=“...‘‘><b>Colored bold text</b></Font> give the same visual result. We did the same 
thing for italic tag. To specify the evaluation conditions, we can see that the evaluation 
logic is similar in all the above sets and it corresponds to the following pseudo condi- 
tion: EXIST(FgColor, bgColor, Alternative) where alternative can be one of the HTML 
tags: b, I, Font.face, and Font.size. The XML form of this condition would be: 

<Meta_Condition MC_ID="ColoredlnfoModel"> 

<Meta_Vars> 

<Meta_Var Name="BgColor" Type="Color"/> 

<Meta_Var Name="FgColor" Type="Color"/> 

<Meta_Var Name="Alternative" Type="FITML_Elem"/> 

</Meta_Vars> 

<Model Expression=”Exist(FgColor, BgColor, Alternative) AND IN(Alternative, AltList)”/> 
</Meta_Condition> 

where EXIST is a predefined GDL function. AltList is the user value given as follows: 
<User_V id=”AltList” type=”Sequence” val=”b I Font.face Font.size”/> 

Notice that the evaluation expression is provided a non structured text. As mentioned 
earlier, this is very simple but requires that the specified text respects the GDL syntax 
for text evaluation expression. After defining the Meta condition we instantiate it on 
every evaluation set via the mapping rules. We specify similar mappings for the re- 
maining sets. 

<Mapped_Condition SetJD="S1A" Meta_ID="ColoredlnfoModer> 

<MetaJVIapping Meta=" BgColor" lnstance="E1"/> 

<MetaJVIapping Meta="FgColor" lnstance="E2"/> 

<Meta_Mapping Meta="Alternative" lnstance="E3"/> 

</Mapped_Condition> 

<Mapped_Condition SetJD="S1B" Meta_ID=" ColoredlnfoModel "> 

<MetaJVIapping Meta=" BgColor" lnstance="E1"/> 

<Meta_Mapping Meta="FgColor" lnstance="E2"/> 

<MetaJVIapping Meta="Alternative" lnstance="E3"/> 

</Mapped_Condition> 
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5.2 Evaluation Optimization 

Decomposing the evaluation process into independent steps in the phases as described 
above offers the possibility for optimising evaluation at each of them. 

Structuring step. As parsing web pages is based on evaluation sets and evaluation 
conditions defined in this step, we can optimise the evaluation at two levels: for a sin- 
gle guideline, there are two ways: identifying the minimum ensemble of sets needed to 
evaluate the targeted guideline, and expressing conditions in the most forward way to 
minimise the number of operations that evaluation engine would need to execute them. 
At the level of many guidelines, we can optimise evaluation by identifying common 
structures or sub-structures. This optimisation cannot be neglected since guidelines are 
expressed at a high abstraction level, and as they come from different sources, it is 
very possible to have guidelines that are totally or partially semantically identical. 

Parsing step. The first significant optimization at this step is the use of the concept of 
exclusion among evaluation sets. By definition, one evaluation set Excludes one 
(many) other evaluation set(s) if its presence excludes its evaluation. This concept is 
based on the Scope concept related to HTML elements (tags and attributes). Gener- 
ally, the excluding set has an element whose scope is within the scope of an element of 
the excluded set. Of course, these two elements must have the same rendering effect. 
For example (Fig. 3), in the context of text color evaluation, a set containing the at- 
tribute Table. bgcolor (like S 1 = { Body. text. Table. bgcolor}) excludes a set containing the 
attribute Body. bgcolor (like S2={ Body. bgcolor. Body. text}), because the scope of Ta- 
ble. bgcolor is within the scope of Body. bgcolor. The second optimisation is to combine 
parsing and evaluation in one step. This means that an evaluation condition is trig- 
gered as soon as an instance of the associated evaluation set is completely detected in 
the evaluated web page. This combination would be optional because, in some situa- 
tions like the need for a detailed evaluation report, it is desired to capture all instances 
of evaluation sets (even non completed or negative ones). 
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Fig. 3. Scope of Table. bgcolor is within the scope of Body. bgcolor 



Evaluation step. The optimisation that can be done at this step relies mainly on opti- 
mising the execution of evaluation conditions. We introduced the concept of basic 
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condition to identify similar or identical parts of evaluation conditions. For example, 
in lnter_GDL 1 of our example, the evaluation conditions for SI (Condi) and S2 (Cond2) 
are very similar (same Body. bgcolor). As Condi will be executed before Cond2, the re- 
sults of executing Condi can be kept if another instance of Cond2 is met with Font. color 
having the same value of Body. text. Fig. 4 shows how multiple instances of foreground 
and background colors can be evaluated positively (the color combination is one of 
the recommended ones), negatively (the color combination is not recommended at all), 
or unknown (the color combination does not belong to any recognised color pattern). 
In Fig. 4, we can observe that 

• In instance ! 1, Table. bgcolor has no effect because TD. bgcolor overcomes it, but we 
captured Instancel of Set3 because we did not considered TD in our evaluation sets. 
This is very easy to repair by modifying the formal structure. 

• In instance !2, it useless in this example, because this combination has no effect. 

• In instance !3, Set2 excludes Setl and Set3, thus the instance of Font.color did not 
cause the creation of instances of Setl or Set3. 
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6 Conclusion 

This paper presented an approach for optimizing automated evaluation of Web U&A 
guidelines based on the concepts of evaluation sets and conditions. This approach 
would present some advantages over approaches adopted by existing U&A evaluation 
tools: 
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• Targeted guidelines: traditional evaluation tools cannot evaluate any guideline 
outside the precompiled set of guidelines hard coded in the evaluation engine of 
the tool. As for a tool adopting our evaluation approach, its main distinctive fea- 
ture is its capability to enable the evaluation of any evaluable guideline. A guide- 
line is said to be evaluable if we can find HTML elements that reflect its seman- 
tics (e.g., the foreground and background colours) and if we can specify the 
needed evaluation conditions using the vocabularies provided by the evaluation 
tool. Thus, such a tool should at least be capable of evaluating guidelines that are 
evaluable by existing tools. 

• Improvement of the evaluation process: using the same methodological frame- 
work to structure all guidelines enables us to obtain non conflicting structures: the 
structuring would show common evaluation sets and common evaluation condi- 
tions if some exist. Partially similar or conflicting guidelines can be identified as 
well. In this way, no guideline is evaluated twice and no evaluation condition will 
be checked more than once. We can also combine parsing and evaluation steps to 
stop evaluating a guideline if one of its evaluation conditions is not verified. This 
is useful for guideline checking, where there is only a need to know whether the 
guideline is verified, as opposed to guideline checking, where we want to know to 
what extent a guideline is respected. 

• Flexibility of the evaluation process: separating evaluation logic from evaluation 
engine in independent phases gives many new evaluation possibilities like choos- 
ing to evaluate a part of a guideline by using a sub-set of its evaluation sets, 
choosing to evaluate particular HTML elements (e.g., images, tables) by selecting 
guidelines that have these elements in one of their evaluation sets, etc. 

• Customisation of evaluation reports: the flexibility of our approach should al- 
low us to generate a custom evaluation report (e.g., a possible simple format is 
given in Fig. 4). In addition to traditional guideline-based evaluation reports gen- 
erated by existing evaluation tools, we should be able to generate reports based on 
objects (images, fonts), customised error messages, etc. 

• Identification of conflicts and similarities among guidelines: expressing guide- 
lines in a logical and structured form allows us to identify potential conflicts 
and/or common elements among guidelines. 

• Guidelines Management: at anytime, a guideline can be added, removed or 
modified without consequence on the evaluation engine. This independence al- 
lows the evaluation system to import new sets of guidelines from outside into the 
tool repository. 
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Abstract. Internet privacy policies are complex and difficult to use. In the eyes 
of end-users, website policies appear to be monolithic blocks of poorly 
structured texts that are difficult to parse when attempting to retrieve specific 
information. In an increasingly privacy-aware society, end-users must be able to 
easily access privacy policies while navigating a website’s pages and readily 
understand the relevant parts of the policy. We propose a structured 
methodology to improve web design and increase user’s privacy awareness. 
This systematic approach allows policy makers to effectively and efficiently 
reshape their current policies by structuring policies according to the subject 
that is relevant to specific user interaction contexts, making them more user- 
centered and user-friendly. The methodology is built upon prior work in privacy 
policy analysis and navigation context design. 



1 Introduction 

Privacy has become a more and more important issue and has recently received a lot 
of attention from consumers, government officials, legislators, and software 
developers due to concerns about increasing personal information collection from 
customers, information disclosure to third parties without user consent, and 
information transfer within and across organizations [5, 7, 8, 9], 

Nowadays, most companies and organizations have posted one or more privacy 
policy documents on their websites. A privacy policy is a comprehensive description 
of a website’s practices on collecting, using and protecting customer information. A 
privacy policy should define what information is collected and for what purpose, how 
this information will be handled, stored and used, whether customers are allowed to 
access their information collected by the website, how to resolve privacy-related 
disputes with this website, etc [6], 

Unfortunately, current privacy policies published on websites are usually long and 
increasingly complex and difficult for users to understand. Research has found that 
many online privacy policies lack clarity and most requires a reading skill 
considerably higher than the Internet population’s average literacy level [1]. There is 
a need to improve the current web design to help Internet users better navigate and 
understand website privacy policies and increase users’ privacy awareness. 
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We further elaborate this problem from the following four aspects of privacy 
policies: (1) content, (2) structure, (3) navigation, and (4) accessibility. 

1 . Content. The language used in privacy policies is often difficult for users to 
understand (e.g. either too technical or too legal), thus preventing them from 
easily understanding the benefits and potential threats entailed by the 
submission of their personal data. As recent studies have demonstrated [1], 
website privacy policies are often ambiguous and conflicting, and therefore 
preventing users from understanding how their personal information will 
actually be treated. 

2. Structure. Different websites use different ways to present their privacy 
practices to users. For example, some policies (see www.bn.com) firstly 
explain what information they collect, and then how the organization will use 
and share this information. Other policies (see www.buy.com) tell where on 
the site they will collect user information and then focus on the strategy and 
technology used to protect that information. Other sites (see 
www.amazon.com) organize their policy’s content with a list of frequently 
asked questions (FAQs), abruptly varying from very general issues (such as 
the kind of information collected) to technical details (e.g. the use of 
cookies) in the attempt to promptly answer the recurrent issues raised by the 
website’s customers. In most of the cases, whichever strategy is chosen to 
organize the content of the privacy policy, the structure presented to users 
takes the shape of a long document with several sections (sometimes split 
into different physical web pages). Putting all of a privacy policy’s 
information into one document may be useful to get a general overview of 
the site’s privacy practices, however, having such a structure, policy texts are 
generally difficult to be contextualized into usage scenarios (e.g. inserting 
credit card information while buying a product) in which users may be 
concerned about the treatment of specific data (e.g. protection and storage of 
credit card number). 

3. Navigation. With a monolithic structure such as this, privacy policy 
navigation is context-independent: wherever a user is navigating on the site, 
she can only access the entire privacy policy document as it is. No matter 
what the user is doing on the site, the policy always tells the same story in 
the same order. The question is, is that really what the user needs? For 
example, if a user connects to a site and realizes that she is promptly 
recognized personally by the site as a returning customer (e.g. “Hi <user’s 
name>, here are our recommendations for you), she may wonder how (and 
for how long) her session data and shopping habits are stored and used by the 
organization. To reach such information, the user must go to another page 
and read a long, and possibly confusing document explaining in very general 
terms the importance of privacy, the effort spent by the organization to 
protect personal data, the technology used, and the conditions of use of her 
personal data (any of which may or may not be relevant for the described 
context of use), etc. Because of this, it is clear that users in a situation like 
this are presented with significant hurdles to retrieve the information they are 
interested in, and thus will more likely make the decision to blindly proceed 
with their site visit, being uninformed as to how their personal information 
will be used. 
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4. Accessibility. The accessibility of privacy policies is also usually very poor. 
The link to the privacy document is often difficult to spot, many times being 
designed as a recurrent pattern, in small font at the very bottom of the page. 
Even if it is accessible from every page of the site, it is not relevant to 
specific website pages. Once accessed, the privacy policy is still difficult to 
parse when attempting to retrieve specific information, as discussed in the 
previous bullets. 

Taking these four dimensions of the problem into account, we can now formulate 
more clearly the specific problem we wish to address in this work. 

In general, concerning the requirements for a “usable” online privacy policy, we 
argue that users should be assisted while browsing or shopping on the site by 
understanding the privacy issues relevant to the current context of interaction. More 
specifically, we should provide users with direct access to relevant portions of the 
privacy policy concerning the information exchanged within the current context. We 
propose that instead of “tell me the whole story about this organization’s privacy 
practices”, websites policies should “tell me now how the site treats my data that I’m 
now concerned with”. 

This paper proposes a systematic approach, which is built upon an existing goal- 
based policy analysis method, to address the aforementioned issues. Our approach 
allows policy makers and web designers to reshape their current privacy policies 
according to subject matters, thus meeting the specific needs relevant to the contexts 
of users’ interactions. The expected benefits of applying this structured methodology 
for policy design are in two aspects. On one hand, users can have a better 
understanding of websites’ privacy practices by accessing the relevant information 
quickly and increase their privacy-awareness. On the other hand, websites can build 
the trust from end-users by specifying privacy policies in an easy to access, easy to 
understand way to satisfy users’ specific privacy needs. 

The remainder of the paper is structured as follows: Section 2 discusses relevant 
previous work on goal mining, a powerful technique to examine and analyze privacy 
policy content. Section 3 details the proposed methodology and the expected results 
of the approach. Examples of application and concrete results are presented in section 
4. Finally, section 5 summarizes the method, discusses some limitations of this 
approach and outlines our plans for future work. 



2 Related Work 

Privacy policy analysis has not been paid enough attention until recently. Studies 
following a structured approach to privacy policy analysis led to the development of 
specific analysis techniques based on goal-oriented requirements engineering 
practices [1, 2, 3, 4]. These conceptual methods and tools, which are based on goal- 
mining, turned out to be particularly effective for examining website privacy policies. 

Goals are objectives and targets of achievement for a system in requirements 
engineering. In the case of privacy policies, a goal describes a statement expressing a 
privacy practice having a coherent and unitary meaning. Goal mining refers to 
extracting goals from data sources (in this case, privacy policies) by applying goal- 
based requirements analysis methods. The extracted goals are expressed in structured 
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natural language in the form of “VERB object” such as “COLLECT site usage 
information”, “PREVENT storing credit card information using cookies”, etc. 

To identify goals, each statement in a privacy policy is analyzed by asking, “ What 
goal(s) does this statement or fragment exemplify ?” and/or “What goal(s) does this 
statement obstruct or thwart? ” 

All action words are possible candidates for goals. Goals in privacy policies are 
thus also identified by looking for useful keywords (verbs). The identified goals are 
worded to express a state that is true, or the condition that holds true, when the goal is 
realized. Consider Privacy Policy #1 from the “Bank of America” Privacy Policy 
(www.bankofamerica.com): 



Privacy Policy #1 : Employees are authorized to access customer information only 
when they need it, to provide you with accounts and services or to maintain your 
accounts. 



By asking the goal identification questions, we identify the goal G144: PROVIDE 
access to Cl ( Customer Information ) to authorized personnel with authorized roles 
from Privacy Policy #1. 
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Fig. 1 . Goal mining. 



Figure 1 shows how a privacy statement is decomposed into four basic components 
of a privacy goal during the goal-mining process: actor, action word, subject type, 
and conditions-constraints-circumstances. The actor represents the stakeholder 
responsible for the goal to be achieved; the action word represents the type of activity 
described by the statement; the subject type describes the kind of user information at 
issue; finally, a goal usually recounts the conditions under which the goal actually 
takes place, the constraints to be respected, or other circumstances that provide the 
context to establish the scope of the goal. 

The goal-mining process and the subject classification serve as the basis of the 
method proposed in this paper. This process of discovery, decomposition and 
representation cannot be entirely automated because it requires significant semantic 
content analysis. Goal-based analysis is best carried out by a team of analysts who do 
not simply chop each statement in a policy into pieces, but who carefully extract each 
statement’s meaning, thus specifying goals that truly reflect the meaning of the 
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original document. However, tool support can greatly enhance the efficiency of the 
goal mining process 1 . 



3 Crafting Usable Policies 

Based on previous discussion, we believe that users may benefit from directly 
accessing the corresponding parts of the privacy policy that are relevant to the web 
page a user is currently on. In this section, we propose a method to structure website 
privacy policies according to the subject matters and make them easily accessible to 
users. The goal of the proposed methodology is to make the transition from 
monolithic, poorly structured privacy documents to agile units of privacy policy 
content relevant to the current usage scenarios. 



3.1 A Process Overview 

We have identified five steps that will help designers create context-dependent 

policies (see Figure 2 for an overview): 

1. Analyze existing privacy policy and identify privacy goals (goal mining) 

Goal mining is the first step of this process and it enables analysts to gather a 
repository of privacy goals that represents the organization’s privacy policies. This 
process is shown as step 1 in Figure 2. The process, techniques and heuristics to 
extract privacy goals from policy statements were detailed in [2, 3, 4], 

2. Organize goals by subject type 

These goals may be organized according to different criteria. For example, goals 
may be clustered by actor, action word and subject type. Organizing goals by 
subject type, which describes the kind of user information that a goal concerns, 
appears to be a very promising strategy for our purposes. Examples of subject 
types are, for example, PII (Personal Identifiable Information), credit card 
information, session data, purchase history, shipping data, account data, user 
preferences, usage habits and shopping habits, authentication information (e.g. 
user name and password ), etc. Other more domain-dependent subject types may be 
explored by analyzing policies from different application domains and business. 
For banking websites, for example, “bank account information” is particularly 
relevant, whereas it would not be relevant for B2C (business-to-consumer) type of 
e-commerce websites. In this step, we structure the collection of goals produced in 
step 1 according to the subject type. Each subject type is associated with a set of 
goals. It is noted that a goal could belong to more than one subject type. This 
process is shown as step 2 in Figure 2. 



1 In our approach, extracted goals are then documented in our Privacy Goal Management Tool 
(PGMT), a web-based tool developed at North Carolina State University. PGMT maintains a 
goal repository for analyses of policies and other documents from which goals can be 
derived. Each goal is associated with a unique ID, a description, a responsible actor, its 
sources and a privacy taxonomy classification. 




36 



D. Bolchini et al. 




3. Create a node for each goal set 

Each set of goals (having the same subject type) may be compiled into a 
navigational node, a micro policy web page recounting the privacy goals in natural 
language. 

4. Identify one or more contexts of user interaction that are relevant to a subject 
classification and associate them with the navigational node concerning that 
subject matter. 

This is a crucial step, in which designers and policy makers must put themselves in 
user’s shoes and envision the usage scenarios in which a user may need 
information about the organization’s privacy policies (e.g., opening an account, 
purchasing a product, accessing personal information, modifying personal profile, 
etc.). Two lines of inquiry that may lead this task are: 

a) “For which task will a user need privacy policy information?” The 
scenarios identified by answering this question usually take place in a 
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given context of interaction, i.e., while a user is browsing a certain set of 
web pages. Therefore, the second line of inquiry becomes relevant. 
b) “Where in the site will a user need privacy policy information to 
accomplish his/her task”? For example, on the page where a user is 
entering credit card information or on the set of pages concerning the 
selection of shipping and payment preferences. 

Thus, the result of this step is the identification of contexts of interaction where 
users may need privacy information (the when and the where). 

Of course, each context of interaction should be associated with the privacy policy 
content relevant for users to accomplish the task in a given context, and since we 
have created a navigational node for each potentially relevant subject type (see step 
3), we can associate one or more goal sets with each navigational context 
identified. 

5. Create a link from each page of a navigational context to the associated goal 
set(s). 

Once an interaction context has its goal sets associated with it, it is necessary to 
create links from the pages of the interaction context to the pages of the associated 
goal sets. Once this is done, a user may easily and directly navigate from a given 
page to the policy information relevant to the task she is doing. The relationship 
between goal sets and context of interaction is bidirectional. On one hand, an 
interaction context (e.g. shopping cart) may be associated with multiple goal sets 
(e.g.- the privacy goals concerning “buying history” and the goals concerning “user 
buying preferences”). On the other hand, the same goal set (e.g. goals concerning 
“buying history”) may concern several interaction contexts, such as “access to 
homepage” (where recommendations are provided on the basis of user’s buying 
habits), “shopping cart” (where related items are provided), “wish list”, 
“customized pages”, etc. 

Links to privacy policy micro-pages should be clearly visible and easily accessible 
to users while they are performing a task (i.e., not at the very bottom of the page in 
small font). 



3.2 Modeling Expected Results 

We now discuss the expected outcomes of this process. Consider a navigation context 
such as the “Purchase process” in a generic e-commerce website. After having 
selected one or more products to buy, user is typically guided through a number of 
steps to complete the transaction. Each step is usually setup in a navigational node (an 
individual web page in this case). In some of these nodes, the user is asked to enter, 
confirm or modify the information concerning the transaction: in one page, the user 
has to enter payment information (credit card number, expiration date, type of card, 
name of the cardholder); in the subsequent page the user has to enter shipping 
information such as full address for delivery, delivery methods, and so on. 

Currently, if the user wants to know more about the collection, storage and the 
security of payment information when he is on the “payment information” page, he 
has to scroll down to the bottom of the page, spot the small “privacy notice” link and 
start reading the long policy document while trying to find some clues and keywords 
to reveal the content of interest. 




38 



D. Bolchini et al. 



To overcome this problem, in our approach, a clearly visible link (for example with 
a text anchor like “privacy of payment information”) is placed on the “payment 
information” page and leads the user directly to the relevant privacy micro-page (e.g. 
a side page unit or a pop-up) telling the user how payment information is handled and 
stored (see Figure 3). 




Fig. 3. Contextualized policy for the “Purchase process”. 



Additionally, in our approach policy may not only be made more accessible to the 
user, but the site might also raise awareness in the user about less-evident privacy 
practices, such as organization’s privacy practices on session data. It is the case, as 
mentioned before, that the data about the user sessions (such as session time, IP, type 
of browser, information stored in cookies) are often associated to PII (Personal 
Identifiable Information) such as name, email, address, etc. Providing direct access to 
privacy information about these kinds of data helps raise user’s privacy awareness on 
protecting their PII. 

Likewise, on the homepage (which is often personalized through the use of session 
data) it may be relevant to have links to policy micro-pages concerning the treatment 
of session data. This may help users understand the reasons why the site gets 
increasingly customized as users access and provide information on various pages of 
the site, and for what other purposes this information is used by the organization. 

The previously presented scenarios are intentionally generic to highlight the wide 
scope of applicability of the proposed methodology. The next section will focus on 
application examples taken from a well-known e-commerce website, thus defining 
more specific solutions and emphasizing the benefits for the user experience. 



4 Application Examples 

To demonstrate our approach, we now present some examples taken from an analysis 
of Amazon.com [10], We have chosen this application because Amazon.com is a 
successful and typical e-commerce website familiar to most Internet users, which has 
a quite complex privacy policy and which may really benefit from adopting a 
contextualized perspective on its privacy communication. Although this is a specific 
case, most of the situations presented are likely common to many websites that gather 
user data for online transactions and better communication with their customers. 
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For each relevant interaction context, we will present contextualization solutions 
by detailing the following aspects: 

• Interaction context: web pages where the user may need specific and 
relevant policy information 

• User issues: possible concents of the user while navigating in the 
interaction context 

• Link to relevant policy: a link available to the user pointing to relevant 
privacy policy micro-pages 

• Policy micro-page content: the specific privacy information contained in 
the linked micro-pages (in terms of relevant privacy goals) 

The discussion of the examples is based on the assumption that web designers have 
applied our methodology described in Section 3 and produced sets of privacy goals 
according to the subject matters. 

Scenario 1: 

Let us consider the scenario in which a first time user of Amazon (a non-customer) 
connects to the site and declares her interest in a product by putting it in the shopping 
cart and then proceeds to check out. At this point, Amazon asks the user whether she 
is already an Amazon customer or not. The registration page for a new customer is 
shown in Figure 4. 




Fig. 4. Registration page for a new customer. 



On this web page, a privacy-aware user might ask this question: how will Amazon 
use my name, email, date of birth and password? Currently, to clarify her issues the 
user has to know that at the very bottom of the page there is a link “Privacy Notice" 
that opens a page starting with the following section “What Personal Information 
About Customers Does Amazon.com Gather?” and then goes on with “What About 
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Cookies?” First, it is unlikely that a first time user (especially a non-frequent web 
surfer) knows a priori that by scrolling to the bottom of the page she will find the 
privacy policy link. Secondly, the privacy policy, as it is, presents the user with 
information that is completely irrelevant to her current context of use: the user already 
knows what kind of information Amazon is collecting from her; moreover, details on 
cookie use are not of interest to the user here. In the other two scenarios described in 
this section, there exists similar situation where the user has to scroll down to the very 
bottom of this page, find the small “Privacy Notice” link, and then read a long privacy 
policy statement to find the relevant information she needs to know. 

Now let us examine this situation using our approach. 

Interaction context (Figure 4): within the registration process, the user is on the 
new customer registration page. 

User issues: “How will Amazon use my name, email, date of birth and password?” 

Link to relevant policy: a link called “see how we treat your registration 
information” or “privacy for data exchanged in this form,” positioned right beside or 
below the form. 

Policy micro-page content: all privacy goals with subject matter {registration 
data}. Amazon’s privacy policy contains the following privacy goals about this 
subject: 

• G1349 : ALLOW customer to access personally identifiable 

information (including name, email, password, etc.) 

• G1338: AVOID companies and individuals who perform 

functions on our behalf using customer personal 
information for other purpose other than performing the 
specified functions 

• G72: AVOID selling customer information to others 

• G 74 s: COLLECT information (e.g. personally identifiable 

information, assets, income, investment objectives, etc.) 
from forms submitted by customer (e.g. applications) 

• G1339 : EMPLOY other companies and individuals to perform 

functions (such as processing credit card payments) on 
our behalf using customer information 

• Gas: GUARD data during transmission using SSL encryption 

technology 

• G1350: SEND customer offers on behalf of other business 

without giving them name and address 

• G639: SHARE customer information among subsidiaries 

• G168: SHARE customer information related to your 

transactions with corresponding affiliates 

• G492: SHARE customer information as permitted by law 

• G1132: SHARE customer information with other organizations 
with customer consent 

• G1351: TRANSFER customer information as assets in case of 
buying/acquiring other companies or being acquired 

Scenario 2: 

In a different scenario, an existing customer may put products in the “Shopping Cart” 
while browsing the product catalog. 
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As soon as a product is put in the Shopping Cart, the user is presented with a page 
displaying suggestions of other potentially interesting products to put in the cart 
(Figure 5). Let us examine this situation. 

Interaction context (Figure 5): Shopping cart suggestions. 

User issues: “Amazon suggests several additional items for me to consider 
according to the purchase habits of other customers. For what other purposes will 
Amazon use information about my purchase habits?” 

Link to relevant policy: a link called “privacy of your purchase history”, 
positioned right below the page title “Customers who shopped for... also shopped 
for...” 







Fig. 5. Amazon Shopping Cart suggestion page. 



Policy micro-page content: all privacy goals having as subject matter: {user 
history or previous purchases, etc } . 

Amazon’s privacy policy contains five goals about this subject: 

• Gi34B : ALLOW customer to access recent product view history 

• G1347: ALLOW customer to access recent purchase history 

• G1344: ANALYZE purchase history 

• G487: COLLECT information about customer online account 

(e.g. balances, transactions, email, bills, payment 
history) 

• G 1346 : USE cookies to store items in your shopping cart 
between visits 
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Scenario 3: 

Finally, let us consider one of the most debated issues in the treatment of consumer 
privacy: the handling and use of credit card information. Figure 6 shows the page 
where the user has to enter payment information such as credit card details. 

Interaction context (Figure 6): “Shipping & Payment” page of the purchase 
process. 

User issue: “Flow will Amazon use and protect my credit card information?” 

Link to relevant policy: a link called “Privacy of credit card information” 
positioned right beside the question “Paying with a credit card?” (see Figure 8). 




Fig. 6. Exchange of sensible payment information. 



Policy micro-page content: all privacy goals having as subject matter: {credit 
card or user payment information} 

Amazon’s privacy policy contains the following privacy goals about this subject: 

• G1337 : ALLOW customer to access payment settings (including 
credit card information, etc.) 

• G1338 : AVOID companies and individuals who perform 

functions on our behalf using customer personal 
information for other purpose other than performing the 
specified functions 

• G37 : COLLECT credit card information for billing 

• G1339 : EMPLOY other companies and individuals to perform 
functions (such as processing credit card payments) on 
our behalf using customer information 

• Gas: GUARD data during transmission using SSL encryption 

technology 
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• G1341: REVEAL only the last five digits of your credit card 
numbers when confirming an order 

• G1342: USE credit history information from credit bureaus 
to help prevent and detect fraud 

• G1343: USE credit history information from credit bureaus 
to offer credit or financial services to customers 

Once selected, goals may be properly rephrased from their formal structure to a 
fluent narrative to make them more understandable for the user. 



5 Summary and Future Work 

Most online privacy policies are poorly structured, hard to understand, long 
documents that do not satisfy end-user’s need for a concise policy statement for 
specific context. In this paper, we present a new method to analyze privacy policies to 
produce privacy goals, and then structure these goals according to the subject matters. 
By doing this, web designers can associate the context of web page to appropriate 
privacy goal sets that concerns the current subject. 

This method is based on a structured and validated policy analysis process. This 
ensures the completeness and consistency of the goal sets displayed in policy micro- 
pages. 

Both users and organizations can benefit from applying the proposed methodology 
in web design. 

For users, they quickly get direct access to the relevant policy information at the 
right time (i.e. when they need it). This enhanced accessibility makes policies more 
and more visible to the users, thus raising overall awareness of Internet users to many 
privacy concerns. It also helps users evaluate the privacy practices declared by the 
organization in a much more straightforward manner. 

Applying the proposed methodology (or even simply adopting the general idea), 
organizations can more easily evaluate the coverage of their privacy policy. By 
analyzing the different interaction contexts, site stakeholders have the opportunity to 
verify whether or not their policy contains information relevant for the user in that 
given context, not just generic and essentially useless information about privacy. A 
contextualized policy also builds trust of users to websites, since it communicates 
more clearly with site privacy practices, showing attention to the concrete needs of 
the user. Finally, contextualizing policies means enhancing the user experience on the 
site, providing more (or less) reasons for visitors to become customers. 

The methodology we proposed and results gathered so far are even more useful for 
multi-channel applications, which are increasingly available on a variety of smaller 
devices, such as PDAs, handhelds, and smart phones. The visualization and 
interaction requirements of such devices pose more strict constraints to user’s 
capability of interacting with and reading long documents such as privacy policy. In 
these cases, the contextualization and design of agile mini-policies are very important 
to make privacy policy really usable to end-users. 

The method has yet to be empirically validated on large scale and across different 
domains through the validation of prototypes and usability testing. Moreover, return- 
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of-investment (ROI) for this change in policy communication still needs to be 
evaluated. 

For future work, we are looking for other relevant interaction contexts where 
contextualized privacy policies can play a key role in improving user experiences. We 
are going to apply this methodology to other domains, such as banking and financial 
institutions websites, which we have already conducted goal-mining and privacy 
policy analysis studies. 

One further extension of the approach that may be explored is that semantic 
associations between goal sets may further enhance privacy policy usability. For 
example, the “session data privacy” micro-page may be linked to the 
“recommendation system” privacy micro-page used by the organization, which 
exploits session data. Similarly, the “Recommendation system privacy” micro-page 
may be linked to the “shopping habits privacy” micro-page, since recommendations 
are built on previous shopping habits of the user, and so on. Such design solutions 
may lead to a privacy policy whose agile navigation highlights even more the 
semantics underlying the privacy practices. 
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Abstract. Many web sites are designed and established without sufficient pro- 
fessional skills and resources. The quality of these websites is often dubious. 
Therefore, how to evaluate the quality of a web site has become an important 
issue. In this work, the stepwise regression method is applied to assess the 
ranking of web sites for two different categories. The ranking is based on the 
average number of visits per day. A total of fourteen factors frequently found 
in the literature are considered as independent variables in developing the 
model. The regression analysis result shows that the regression models differ 
for two different categories of web sites, but there are three variables common 
to the two resulting models. Latest update. Broken links, and Help features. The 
average prediction accuracies of both models exceed 75%. 



1 Introduction 

Constructing a web site are nowadays straightforward, demanding few technological 
efforts. Besides professional developers, there are more and more amateurs engaged 
in the production of many web sites [1], Although the number of sites flourishes ex- 
tensively, the content and the quality of some sites do not improve simultaneously. 
Therefore, how to evaluate the quality of a web site has become an important 
issue [4], 

A web site should be evaluated from both the content and the design of the site 
[5], The content covers the content of an entire site as well as an individual web page. 
The characteristics of the content for the entire site include not only the information 
content, but also support for transaction and elements of entertainment [1], The con- 
tent of a web page comprises of elements of text, color, graphics, mages, audio, ani- 
mation, and so on. The criteria for assessing the content include correctness, periodi- 
cal update, completeness, organization and clearness, attractiveness, value [3]. 

The design of a web site consists of the interface or the layout design of web pages 
and the navigational design of web sites. It could be evaluated in four aspects [6]: 
usability, functionality, reliability, and efficiency. Usability concerns how to assist 
users in using the site effectively. Functionality includes search and navigational 
mechanisms. Reliability addresses the correctness of the link and the errors incurred 
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by different configurations. Efficiency tackles factors that might affect the download 
speed of web pages and the accessibility of the information on a page. Ivory and 
Hearst evaluate web pages with 1 1 measures covering both the content and the design 
aspects [2]. In this work, the stepwise regression method is applied to assess the 
ranking of web sites for two different categories in Taiwan. The ranking is based on 
the average number of visits per day. Such ranking represents somehow the ranking of 
user satisfaction, and therefore the quality of a web site. 



2 Research Variables 

An indicator of the ranking of web site may be based on how often the site is utilized, 
which reflects true opinions of real users somewhat. Therefore, we base the depend- 
ent variable on the average number of visits per day to a web site during a span of 
three months. Because the number of visits varies enormously for top sites and poor 
sites, we map these numbers into their ranking positions instead of the using the ac- 
tual numbers. The definition of the variable follows: 

E =1 oo-J^xl/-!) 

(«- 1 ) 

Y i is the score of the ;-th ranked in the set of sites, and n is the number of web sites in 
the set under study. 

The possible values for independent values are all scaled to the range from 0 to 
100 for easier manipulation and exploration of the model. There are a total of four- 
teen independent variables under consideration as follows: 

1. Site map: A numerical function X, is defined to be 100, if the site provides a site 
map or a TOC; 50, if the site provides a navigational menu; 0, if none is provided. 

2. Help feature: The corresponding function X 2 is given as 100, if providing organ- 
ized help feature such as FAQ; 80, providing interactive help feature such as bul- 
letin board; 60, providing email and responding within a week; 40, providing email 
and responding more than a week; 0, providing none. 

3. Latest update: We define its function X. to be 100, updated within 3 days; 75, up- 
dated within a month; 50, updated within 3 months; 25, updated within a year; 0, 
otherwise (including no update indication). 

4. Font count: Function X 4 is defined as 100 — 20 x | n — 3|, where n is the number 
of fonts and 1 ^ n ^ 7; 0, otherwise. 

5. Color count: Function X 5 is defined to be 100 — 20 x | n — 7|, where n is the num- 
ber of colors and 3 ^ n ^ 1 1 ; 0, otherwise. 

6. Foreign Language Support: The corresponding function X 6 is defined to be 100, if 
providing both English and Simplified Chinese versions; 80, if providing English 
or Simplified Chinese version, plus another version for other language; 60, if pro- 
viding English or Simplified Chinese version; 0, otherwise. 

7. Search mechanisms: Function X 7 is given as 100, if providing search mechanism 
covering the entire site; 50, if providing search mechanism covering part of the site 
(bulletin board, for example); 0, if providing none. 
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8. Link count: The corresponding function X g is defined as in [6], 

9. Scrolling: Let n be the ratio of the length of the page divided by the length of the 
screen. The function X 9 is 100, if n 3; 100 — 15x ( n — 3), if 4 Je n S5 6; 55 
— 10 x (n — 6), if 7 < n < 8; 35 — 5 x (n — 8), if 9^ n < 10; 0, otherwise. 

10. Word count: We define the variable X 10 as ( n is the total word count) 100, if 651 

^ n < 1300; 100 - 20 x ceiling((n -1300) / 650), if 1301 < 3900; 80, if 

326 <n < 650; 60, if 160 325; 40, if n < 160; 0, otherwise. 

11. Broken link: We adopt the function proposed by Olsina et al. in [6] as X n . 

12. Static page size: Function X 12 is defined as (let the size be n) 100, if n A 80 KB; 
90, if 80 KB < n ^ 100 KB; 100 - 10 x truncate)/! / 50 KB), if 100 KB < n ^ 
500 KB; 0, otherwise. 

13. Image label: The function proposed by [6] is used as X 13 . 

14. Number of panes regarding frames: Function X 14 is the same as that Olsina et al. 
propose [6]. 

Our initial regression model therefore takes the following from: 

Y =b 0 + b X, + b, Y, + b 3 X 3 + b 4 X 4 + b, X - + b 6 X 6 + b 7 X 1 + b g X y + b 9 X 9 + b 10 X 10 + b 13 
i L b 12 X a + b 13 X a + b 14 X 14 



3 Data Analysis 

The source data of this research comes from a web site called HotRank 
( http://www.hotrank.com.tw ). HotRank maintains visit information for several thou- 
sand web sites. Theses web sites are further classified into different categories from 
which two categories, academy and literature and shopping are chosen. The sites that 
appear in the seasonal ranking list from January to March 2003 are chosen as sample 
data. 

To ensure the uniformity of the visit data, we further requires that the chosen sites 
are operational since January 1st 2003. The number of sites in the academy and lit- 
erature category conforming to the condition stated above is 305. However, all sites 
ranked after the 90th position are eliminated because their average number of visits 
per day falls to zero and other sites failing to provide the average number of visits per 
day are also taken out. After reduction, there are a total of 60 sites from the academy 
and literature category. Similarly for the shopping category, the number of sites re- 
duces to 198. We randomly select 30 sites respectively from two categories to con- 
duct regression analysis. Some of the data for the independent variables result from 
average values of the homepage and other nine pages randomly selected from the site. 

Before the stepwise regression analysis, the Pearson analysis is performed and the 
results indicate that all the interdependency values between any two independent 
variables are less than 0.55, so there is no collinear relationship between any two of 
the fourteen independent variables. Therefore, it is appropriate to consider all these 
variables in the stepwise regression analysis. The Durbin-Watson (DW) test is em- 
ployed to examine that the error e. of independent variables should not be self-related. 
From our analysis, the DW values of the models for the academy and literature cate- 
gory and shopping category are 1.476 and 1.589, respectively. This reveals that there 
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is no significant self-relation in the two models and the prediction of the models can 
be trustworthy. Results of the stepwise regression analysis are described in the fol- 
lowing paragraphs. 

F test examines the overall regression model, also called as Analysis of Variance 
(ANOVA). The F values of the analysis result are 20.324 and 20.637 respectively for 
the academy and the shopping categories. Both of the P-Values are less than 0.005. 
Therefore, the linear relationships of our models are well established. 

The coefficient of multiple determinations measures the proportion that independ- 
ent variables are able to explicate the dependent variable. Another related measure is 
the adjusted coefficient of determination, which is considered more representative 
than the coefficient of determination. Applying these measures, the coefficient of 
multiple determination and adjusted coefficient of determination for the model of the 
academy and literature category are 0.765 and 0.727, respectively. As for the shop- 
ping category, the coefficient of multiple determinations and adjusted coefficient of 
determination are 0.768 and 0.730, respectively. This indicates that the independent 
variables in the models can account for around 73 % of the dependent variable. 

Finally, the t test is to examine whether there is a significant linear relationship 
between every independent variable and the dependent variable. If there is no signifi- 
cant linear relationship between an independent variable Xi and the dependent vari- 
able Y, the coefficient of the variable h should be set to zero. Applying the t test to all 
the fourteen independent variables, the result shows that the final regression model of 
the academy and literature category is 

Y = - 58.308 + 0.493 x Last_update + 0.472 x Broken_links + 0.219 x Site_map 
+ 0.475 x Help_feature 

And the model for the shooping category is 

Y = - 46.345 + 0.494 x Help_feature + 0.371 x Static_page_Size + 0.378 x 

Last_update + 0.259 x Broken_links 

The adjusted coefficients of determination of the two models are 0.727 and 0.730, 
respectively. Considering the diversities of web sites, such degree of accounting pre- 
cision is well acceptable. 

In order to check the applicability of our model, another 15 sample sites are ran- 
domly selected for each category from the set of sites that is not selected previously in 
establishing the models. We apply the model to predict the rankings of these sites and 
compare them with the actual rankings. The result shows that the average prediction 
accuracy of the model for the academy and literature category is 76.0 %, and that of 
the model for the shopping category is 79.2 %. 



4 Discussions and Conclusions 

The difference of the independent variables in the two models suggests that there are 
indeed different indicators for different categories of web sites and the effects of these 
indicators are also different. There are three variables common to the two resulting 
models. Latest update. Broken links, and Help features. These three variables address 
different aspects of a site, information, reliability and usability, respectively. This 
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implies a good web site must excel in various aspects. Latest update involves the 
content of the web site and more specifically how often the content of the site is up- 
dated. In a fast changing world, this is certainly a great concern to the site users. Bro- 
ken links address the reliability issue as well as the correctness of the content in a web 
site. It is always frustrating for users to chase after a link only to find it leads to no- 
where. Help features, on the other hand, tackle the usability of a web site. As the a 
web site evolve in functionality and complexity, usability issues commonly found in 
software applications surface inevitably and online help plays an important part in 
usability of a software system. 

The Site map variable address yet another important aspect of a web site, i.e., 
navigation issues. The reason that it is not present in the model of the shopping cate- 
gory is that the content of sites in this category map directly to a hierarchical structure 
reflecting the structure of product catalogue. However, the static page size is a sig- 
nificant factor for these online shopping sites. A page with an immense number of 
bytes would take time to download, which is not contradictory to the efficiency that 
most online shopper expects. While we may imagine users in the sites of the academy 
and literature category to be more leisurely when they surf these sites, thus efficiency 
is not a great concern. 

We apply linear regression method for assessing the ranking of a web site based on 
the number of visits per day. A total of fourteen factors frequently found in the lit- 
erature are considered as independent variables in developing the model. The regres- 
sion analysis result shows that the regression models differ for two different catego- 
ries of web sites. The average prediction accuracy of the model for the academy and 
literature category is 76.0%, and that of the model for the shopping category is 
79.2%. 
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Abstract. This paper illustrates a method and a toolset for quality evaluation 
of Web applications that exploits conceptual specifications, deriving from the 
adoption of model-based development methods, for the evaluation in pre- and 
post- delivery phases. 

Keywords: Conceptual Modeling, Web Application Quality, Web Usage Mining. 



1 Introduction 



The ever-increasing spread of the Web asks for new methods for improving the quality 
of Web applications. Conceptual modeling improves the quality of final applications by 
fostering regularity and the definition and reuse of effective solutions. However, little 
attention has been put on using conceptual specifications for enhancing the evaluation 
activities occurring throughout the whole development process. 

Wa have defined a model-based framework that exploits conceptual schemas for 
evaluating Web applications both at design time and after the application deployment [6, 
7], This paper illustrates some recently introduced components that assist evaluation 
activities performed after application deployment. More specifically, such components 
elaborates Web usage logs enriched through meta-data deriving from the application 
conceptual schema. 

Our evaluation framework has been defined for a specific conceptual model, WebML 
[2], and has been implemented extending a commercial CASE tool [8]. WebML offers a 
set of visual primitives for defining conceptual schemas that represent the organization 
of the application contents and of the hypertext interface. The organization of data is 
expressed though the Entity-Relationship model (E-R). The hypertext is then specified 
by composing elementary pieces of contents retrieved from the database, called content 
units, to form pages. WebML primitives are also provided with an XML representation 
that specifies additional properties, not conveniently expressible in the visual notation. 

This paper introduces the Web log analysis covered by our evaluation framework, and 
shortly describes the architecture of an accompanying tool-set supporting the automatic 
execution of the proposed evaluation method. A deeper description, as well as results of 
applying the quality evaluation method to real-life Web applications can be found in [5], 



N. Koch, P. Fraternali, and M. Wirsing (Eds.): ICWE 2004, LNCS 3140, pp. 50-54, 2004. 
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2 The Evaluation Framework 

Our evaluation framework supports three kinds of analysis, based on the knowledge of 
the application conceptual schema. 

In the pre-delivery phase, the Design Schema Analysis (DSA) verifies the correctness 
and the internal coherence of specifications [3,6]. It operates directly on conceptual 
schemas by looking for design errors and inconsistency. 

In the post-delivery phase, evaluation still exploits the schema knowledge. Thanks to 
an advanced logging mechanism extending the runtime engine of WebML applications, 
Web logs are enriched with meta-data related to the application conceptual schema, thus 
obtaining the so-called conceptual logs [7]. The evaluation then focuses on such enriched 
logs, according to two techniques: 

- Web Usage Analysis (WUA) produces reports on content access and navigation paths 
followed by users. 

- Web Usage Mining (WUM) applies mining techniques for discovering interesting 
(sometimes unexpected) associations between accessed data. The aim is to identify 
possible amendments for accommodating newly discovered user needs. 

The peculiarity of the post-delivery evaluation is the exploitation of conceptual logs, 
defined as XML-based “enriched" Web logs that integrate the conventional HTTP log 
data, generated by Web servers in ECLF (Extended Common Log Format) format, and 
meta-data about the computation of page contents. As can be seen in Figure 1, for each 
requested page such meta-data include: i) identifiers of the page and of its content units, 
as resulting from the application conceptual schema, that provide references to detailed 
properties defined in the conceptual schema but not traced by the logging mechanism; 
ii) primary keys of database instances used at runtime to populate content units. 

DSA has been illustrated in [3,6]. Therefore, in the next subsections we concentrate 
on the two analysis approaches operating over conceptual logs. 



2.1 Web Usage Analysis 

WUA analyzes conceptual logs for computing access reports on user content access and 
user navigation paths. The main objective is identifying the contents most requested by 
users and evaluating if the hypertext design accommodates such user needs. 

WUA comes in two flavors: Access Analysis, and Navigation Analysis. 

Access Analysis computes traffic statistics, with the aim of verifying if the commu- 
nication goals the Web application has been designed for are supported by the hypertext 
interface. The model-based approach, which distinguishes between data modeling and 
hypertext modeling, allows performing: 

- Data Access Analysis: it computes statistics for the access to data schema entities 
and their specific instances. 

- Hypertext Access Analysis: it focuses on the usage of the hypertext elements (content 
units, pages, areas) publishing specific data elements. 
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1 <Logs> 

2 <Request id_P="3178"> 

3 <Page SchemaRef ="page33"/> 

4 <LocalTime> 

5 

6 </LocalTime> 

7 <User> 

8 

9 </User> 

10 <PageUnits> 

11 <Unit> 

12 <Unit_Id SchemaRef ="data_unit84"/> 

13 <DataInstance>17</DataInstance> 

14 </Unit> 

15 <Unit> 

16 <Unit_Id SchemaRef ="index_unit9"/> 

17 <DataInstance>10</DataInstance> 

18 <DataInstance>5</DataInstance> 

19 <DataInstance>9</DataInstance> 

20 </Unit> 

21 </PageUnits> 

22 </Request> 

23 

24 </LogS> 



Fig. 1. Extract from the conceptual logs. 



It therefore results that our Access Analysis extends the statistics normally offered 
by state-of-the-practice traffic analyzers, which mostly address page visits, and do not 
log database instances populating dynamic pages. 

Navigation Analysis then verifies if the hypertext topology supports content acces- 
sibility. It reconstructs navigation paths adopted by users for reaching core application 
contents, with the aim of identifying if end users exploit navigation paths embodied 
within the designed hypertext, or else adopt alternative access mechanisms. The re- 
construction of the user interaction results to be precise and detailed, as it exploits the 
conceptual schema of the application hypertext. Also, reconstructed user paths, as well 
as the identified problems, are represented on top of the visual specification of the con- 
ceptual schema; this facilitates the comparison between the designed hypertext and its 
actual use by users. 



2.2 Web Usage Mining 

WUM operates on conceptual logs, and applies XML mining techniques [1] for discov- 
ering interesting (sometimes unexpected) associations among visited hypertext elements 
and accessed data. The execution of mining tasks produces: 

- XML association rules of the form X => Y, stating that when the log element X 
(called the rule body ) is found, it is likely that the log element Y (called the rule head ) 
will be also found. Depending on the adopted mining statement, the retrieved asso- 
ciation can be related to database entities or instances, hypertext components (areas, 
pages, content units), or also hypertext components coupled with their populating 
data instances. 
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- XML sequential patterns, whose rule body and head are also bounded to their position 
in the log sequence, thus representing temporal relations. 

Based on such rules, so far we have focused on two mining tasks: 

- Finding data that are often accessed together, considering as transaction a user 
request, implemented through the mining of association rules between data entities 
and instances accessed within the same user session. It is worth noting that such 
associations are not easily discovered in traditional logs that do not record data 
instances used to populate dynamic Web pages, and generally require several post- 
processing efforts. 

- Analyzing user navigation sequences for accessing some core information contents, 
by mining sequential patterns related to sequences of pages and content units within 
the same user session. The WebML characterization of information concepts and 
content units allows filtering sequences, concentrating the analysis on relevant nav- 
igation paths leading to some selected core concepts. 

3 The Framework Architecture 

The software architecture of our evaluation framework is organized in three distinct 
layers: 

- The Data Extraction Layer gathers inputs needed for evaluation (Web server ac- 
cess, logged execution data, the application conceptual schema and the application 
data source), and transforms them into the format required by the three analysis 
techniques. It also generates conceptual logs. 

- The Analysis Layer includes software components for the execution of DSA, WUA 
and WUM over data gathered through the Data Extraction Layer. 

- The Result Visualization Layer allows evaluators to invoke the different analysis 
tasks and shows graphically the analysis results, through a graphical user interface. 

Some XML repositories enable the interaction between layers: 

- The Analysis Data Warehouse stores inputs gathered and elaborated by the Data 
Extraction Layer, represented in XML format. 

- The Result Warehouse stores the results produced by the Analysis Layer in XML 
format. Such data are then used by the graphical user interface for generating and 
visualizing the analysis reports. 

- The Analysis Tasks Repository stores the analysis procedures that can be expressed 
both in XSL and XQuery. 

The ubiquitous use of XML technologies improves the number of strategies the 
evaluator can adopt in order to manipulate and query data. Also, the quality evaluation 
framework results to be very flexible and extensible: new analysis tasks can be easily 
specified and added to the framework. Therefore, each design team can define its own 
quality criteria, code their measures in XSL or XQuery, two extensively used W3C 
standards, and adding them within the the Analysis Tasks repository. Additionally, the use 
of warehouses between layers improves the framework extensibility, since it is possible 
to add new software modules, for example for performing new kinds of analysis, without 
affecting other components. 
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4 Conclusion 

During last years, several methods and tools have been proposed for the analysis of Web 
logs [4], The majority of them are however traffic analyzers. In addition to calculating 
traffic statistics, our Web Usage Analysis is able to compute advanced statistics, related 
to database entities and instances, and to hypertext components of any granularity. 

Thanks to the intensive use of the application conceptual schema, our framework 
introduces a number of advantages also with respect to Web usage mining. Several data 
mining projects have demonstrated the usefulness of a representation of the structure 
and content organization of a Web application [4], 

Our future work will concentrate on the incremental enrichment of the statistics and 
mining tasks for analyzing Web usage data. We are also working on the improvement of 
the graphical user interface, for allowing designers to define new analysis tasks though 
a visual paradigm, without the need of manual XSL and XQuery programming. 



References 

1. D. Braga, A. Campi, S. Ceri, M. Klemettinen, and P. Lanzi. A Tool for Extracting XML 
Association Rules. In Proc. of ICTAI'02, November’02, Crystal City , USA. IEEE Computer 
Society, 2002. 

2. S. Ceri, P. Fraternali, A. Bongio, M. Brambilla, S. Comai, and M. Matera. Designing Data- 
Intensive Web Applications. Morgan Kauffmann, 2002. 

3. S. Comai, M. Matera, and A. Maurino. A Model and an XSL Framework for Analyzing 
the Quality of WebML Conceptual Schemas. In Proc. of the ER’02-IWCMQ’02 Workshop, 
Tampere, Finland, October'02 , LNCS. Springer, 2002. 

4. R. Cooley. The Use of Web Structures and Content to Identify Subjectively Interesting Web 
Usage Patterns. ACM TOIT, 3(2), May 2003. 

5. P. Fraternali, P. Lanzi, M. Matera, and A. Maurino. Model-Driven Web Usage Analysis for the 
Evaluation of Web Application Quality. Technical Report, Polytechnic of Milan, April 2004. 

6. P. Fraternali, M. Matera, and A. Maurino. WQA: an XSL Framework for Analyzing the Quality 
of Web Applications. In Proc. of ECOOP'02-IWWOST’ 02 Workshop, Malaga, Spain, June’02, 
2002 . 

7. P. Fraternali, M. Matera, and A. Maurino. Conceptual-level Log Analysis for the Evaluation 
of Web Application Quality. In Proc. of LA- Web ’03, Santiago, Chile, November’03. IEEE 
Computer Society, 2003. 

8. WebRatio. http://www.webratio.com. 




Using Adaptive Techniques to Validate and Correct 
an Audience Driven Design of Web Sites 



Sven Casteleyn 1 , Irene Garrigos 2 , and Olga De Troyer 1 

1 Vrije Universiteit Brussel, Department of Computer Science, WISE, Pleinlaan 2, 

1050 Brussel, Belgium 

{ Sven . Casteleyn, Olga . DeTroyer } @vub .ac.be 
2 Universidad de Alicante, IWAD. Campus de San Vicente del Raspeig, Apartado 99 03080 

Alicante, Spain 
igarrigos@dlsi .ua.es 



Abstract. An audience driven philosophy for web site design takes the differ- 
ent target audiences as an explicit starting point, and organizes the basic navi- 
gation structure accordingly. However, for the designer it is not always easy, or 
sometimes even impossible, to assess the different requirements of the different 
target audiences correctly. In this paper, we show how to correct for such pos- 
sible flaws using adaptive behavior. A mechanism for detecting both missing 
and superfluous information in a certain user's navigation path, by observing 
user’ s browsing behavior, is provided. The designer specifies possible adaptive 
changes based upon this detection at design time, using a language (Adaptation 
Specification Language) designed specifically to express changes in the navi- 
gation structure of a website. 



1 Introduction 

One approach described in the literature to improve usability [10] of web sites is the 
Audience Driven design philosophy [5] [6] . For a web site design, it takes as a start- 
ing point the identification of the different target audiences, and arranges them in a 
hierarchy according to their requirements. From this hierarchy, the main structure of 
the web site is derived. Concretely, for the visitors this results in links on the home- 
page, each representing a different navigation path for a different kind of visitor 
(called audience track ) containing all information/functionality relevant for that kind 
of visitor. 

Although this design philosophy significantly reduces the amount of information the 
visitor needs to plough through, it can also be a cause of annoyance if the user cannot 
find the information he is looking for in the chosen audience track, or if his track 
contains information of no interest to him. As it is more difficult for web designers 1 to 



1 Target audiences for web sites are often more difficult to access and perform standard re- 
quirements engineering techniques (e.g. questionnaires) upon, compared to classical pro- 
spectus users of a standard application. 

N. Koch, P. Fratemali, and M. Wirsing (Eds.): ICWE 2004, LNCS 3140, pp. 55-59, 2004. 
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assess Ihe exact requirements relevant or irrelevant for a certain user, it is well 
possible that some information fulfilling a requirement is missing in the audience 
track for a certain visitor (but present in another), or other information is put wrongly 
in that track. In this paper, we will tackle this problem by describing how to identify 
such missing or superfluous information for a certain target audience, and how to 
correct for it, by adapting the structure and navigation in the website. The framework 
for this work is the Web Site Design Method (WSDM) [6], an audience driven design 
method. 

Web design methods that have support for adaptation include WebML[3], Hera[7], 
UWE[9] and OOH[8]. However, these methods focus either on personalization (e.g. 
towards different devices, content personalization), or handle a bottom-up approach 
to constructing pages from basic concepts. As far as the authors are aware of, there is 
no other research combining design support for altering an existing structure based 
upon user access, to improve the structure of the site towards all users. 



2 Identifying Missing or Superfluous Information 

Three steps are involved in detecting superfluous or missing information in a certain 
audience track: 

1. Determine to which audience class the current user belongs 

Due to audience driven design, the user has to choose an audience track with 
his first click, thus choosing to which audience class he belongs. 

2. Determine if and which information the user visits, both within and outside 
his audience track. 

As we want to keep track of which information is visited outside a particular 
audience track, and relate this information to the frequency of visits inside the track, 
we cannot just store the (total) number of accesses to every piece of information 
(modeled as a chunk in WSDM). Instead, we need to store the number of visits to 
each piece of information relative to each audience class. This data can be conven- 
iently stored in a matrix, which we will call the information access matrix. Rows of 
the matrix represent the information (chunks) that represents the different elementary 
requirements, while the columns represent the different audience classes. Over time, 
the matrix contains a good summary of the amount of accesses to the different chunks 
for each audience class. 

3. Analyze the accumulated data to determine if superfluous or missing infor- 
mation for a certain audience track is detected. 

We fall back on known statistical techniques: we consider the problem of de- 
tecting missing information as the problem of deciding whether a certain value (i.e. 
the amount of accesses to foreign information, information outside an audience track) 
fits well in a given sample data set (i.e. the set of amounts of accesses to native infor- 
mation, information within the audience track). Although different statistical tech- 
niques can be used, we have chosen in this paper to use median (which is robust) as a 
measure of central tendency combined with median absolute deviation as a measure 
of spread. As most values (see [4] for a more exact estimate) of a dataset probably lie 
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within the distance of the spread from the middle value, we can use (median - MAD), 
calculated over the set of accesses to native information as a threshold value above 
which accesses to foreign information is considered relevant. 

We consider the problem of detecting superfluous information as the problem of 
detecting outliers [1] (i.e. the amount of accesses of possible superfluous information) 
within a given dataset (i.e. the set of amount of accesses to native information). Al- 
though different statistical techniques can be used, we have chosen in this paper to 
use a double strategy to find low values: find values that lay both far below the mid- 
dle value of the given dataset and far from their bigger neighbor. To determine which 
point lies far from the middle value, we again calculate the median and MAD for the 
given dataset, and test which values of the dataset lay below the threshold (median - 
MAD). To determine which values are far from their neighbor, we take the ordered 
dataset, and calculate the distances between each 2 consecutive values. These dis- 
tances give us a new dataset, for which we calculate mean and standard deviation (= 
std). Calculating the interval [(mean-std) (mean+std)] gives us the range in which 
most of the distances (i.e. 50% for normally distributed data) lay. Distances above 
the (mean+std) are thus relatively big, compared to the other distances, and we have 
identified two points that are relatively far from each other. 



3 Clarifying Example 

To clarify this method, let’s consider a real life example of a website (partly) built 
according to an audience driven design philosophy: the NASA web site 
(http://www.nasa.gov/). The information access matrix for these tracks is shown in 
figure 1 . For simplicity, we consider only the audience classes “Media & Press” (with 
native information inf I to inf6 ) and “Educators” (with native information inf7 to 
inf 12). As explained, each cell in the matrix denotes the number of visits to some 
information by a certain audience class. As we were unable to obtain the real access 
information for the NASA website, we have used for this example fictitious data. 

Let’s now analyze the accesses to the native information of the Educators audience 
track (inf7 ... inf 12), and determine if accesses to foreign information (infl ... inf6) 
were significant compared to these accesses: 

Data set (ordered): 10 15 20 30 50 56 

Median: 25 ; MAD: 12.5 ; Threshold: 25 - 12.5 = 12.5 

Detected foreign requirements: Press Release Archive (40 accesses) 

We can thus conclude that this information should somehow be included in the Edu- 
cator track (how and where this information is added is the subject of the next sec- 
tion). Let’s now consider the calculations to identify possible superfluous informa- 
tion in the “Media & Press” audience track. 



Data set (ordered): 16 31 38 40 49 52 
Median: 39; MAD: 9 
Lower limit: 39 - 9 = 30 



Data set of distances: 15 7 2 9 3 
Mean: 7.2 ; Standard deviation: 4.7 
Upper limit: 7.2 + 4.7 = 1 1 .9 
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As the first element (‘Fact Sheets’, 16 accesses) of the original dataset lies below the 
corresponding lower limit (30), and the distance between the first and the second 
element (15) lies above the corresponding threshold (11.9), we can conclude that 
‘Fact Sheets’ is detected as (possibly) superfluous for Media & Press audience class. 





Media & Press 


Educators 


Inf. 1 


Press Release Archive 


52 


40 


Inf. 2 


Press Contacts 


49 


4 


Inf. 3 


Press Kits 


31 


10 


Inf. 4 


Fact Sheets 


16 


5 


Inf. 5 


Speeches 


40 


5 


Inf. 6 


Images 


38 


12 


Inf. 7 


Contacts for educators 


0 


56 


Inf. 8 


Professional development 


5 


50 


Inf. 9 


Student opportunities 


1 


30 


Inf. 10 


Fellowships and grants 


0 


10 


Inf. 11 


Teaching Internet Resources 


3 


20 


Inf. 12 


Teaching Multimedia Resources 


2 


15 



Fig. 1. Information Access Matrix 



4 Correcting Missing or Superfluous Information 

Flaving identified missing or superfluous information in a certain audience track, the 
structure of the web site can be adapted (automatically) to correct the detected defi- 
ciencies. To specify this possible adaptation at design time, we use the Adaptation 
Specification Language (ASL) [2] defined over the navigation model of WSDM. 

Possible actions taken upon detection of missing information within an audience track 
varies from duplicating the information in the place it is missing, provide a link to the 
existing information, or totally re-arrange the structure of the site. The approach 
shown in this paper consists of duplicating the detected nodes, and offering a link to 
the information at the root of the audience track: 

Foreach AudienceClass in Website 

Foreach node not in NodesOfAudienceTrack(AMdienceC/ass): 
if node in Missinglnformatioti(AudienceClass) 
then add L i n l< ( root) A u d i e nccTrackfA udie nee Class ) , duplicat e(node)) 

Information identified as superfluous in a certain audience track does not necessarily 
need to be removed. Although visited only few times, it might still be valuable for (a 
small amount of) visitors. Automatic adaptation is possible (for example, apply link 
sorting), but more tricky and lengthy to describe; due to space restrictions we cannot 
describe it here. For now, we consider the detection of superfluous information rather 
as an alert to the web master, than something that requires (automatic) adaptation. 
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5 Conclusions and Further Work 

In this paper, in the context of an audience driven design method for web sites, 
WSDM, we provide a mechanism for adaptively correcting the structure of the web 
site. By observing the user browsing behavior and the use of statistical techniques, 
missing or superfluous information in a particular audience track are automatically 
detected. At design time, the designer can specify (using the Adaptive Specification 
Language) the adaptive actions that should be taken in case such situations are de- 
tected at run time. By doing so, the structure of the web site will be better tailored to 
the needs of the different audience classes. 

Future work includes accommodating the Adaptation Specification Language to make 
it possible to acquire all necessary information (e.g. creating and updating the infor- 
mation access matrix). Further research in analyzing the data from the matrix is be- 
ing performed. In particular, detecting correlations between (the same) missing or 
superfluous information in different audience classes, and a cleverer way to adapt the 
structure of the site accordingly. Different adaptation strategies upon detection of 
missing/superfluous information are also a way to continue the research. 
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Abstract. Methodologies for the engineering of Web applications typically pro- 
vide models that drive the generation of the hypermedia navigation structure in 
the application. Most of these methodologies and their models consider link fol- 
lowing as the only materialization of the navigation structure. In this paper we see 
how extended user input can dynamically influence the navigation structure. By 
means of Hera it is shown how one can define this extended user input and capture 
the functional aspects related to the hypermedia dynamics in the RDF(S)-based 
design models. For this purpose we discuss the definition of input controls, the 
representation of state information, and the embedding of both in the application 
model. We also present the XML/RDF-based architecture implementing this. 



1 Introduction 

Under the influence of the World Wide Web we have seen the development of a new 
type of (data-intensive) information systems. These so-called Web Information Systems 
(WIS) [1] are characterized by the use of hypermedia navigation through the content 
of the system, in combination with the traditional functions of an information system 
allowing to update and query the content. As examples of WIS applications we mention 
online services like real-estate sales, employee information, museum information, or 
mail order catalogs. 

The engineering of WIS requires different methodologies than the ones than we have 
been using for information system development over the last decades. In the traditional 
approach, used for example in more database-oriented applications, we see that most of 
the engineering activity is related to structuring the data so that the structure matches 
the standard software component, i.e. relational database. The subsequent design of 
presenting the content to the user is considered in the query facility associated with the 
software. On the other hand, with the original hypermedia approaches we see a different 
pattern, since they typically assume a process of manually linking documents. The design 
process centers on the design of the navigation in the presentation of the content in terms 
of a hyperdocument. 

In the engineering of a WIS the designer has a challenging task. On the one hand, the 
designer has to provide the users with all benefits from using the hypermedia paradigm 
and particularly the notion of navigation through the information offered by the system. 
On the other hand, the designer has to support the users in their maintenance of the 
content by allowing updates and queries to the data. Many of today’s data-intensive Web 
applications show the designer’s attention for the maintenance of the data, but at the 



N. Koch, P. Fraternali, and M. Wirsing (Eds.): ICWE 2004, LNCS 3140, pp. 60-73, 2004. 
(c) Springer- Verlag Berlin Heidelberg 2004 




Modeling User Input and Hypermedia Dynamics in Hera 



61 



same time they show the risk of losing those benefits of hypermedia that have been the 
foundation for the success of the Web. 

In the research field of WIS engineering we have seen proposals for methodologies 
that extend and improve the methodologies for manual hypermedia design for appli- 
cation in data-intensive information systems: we mention as representatives RMM [2], 
WebML [3], OOHDM[4], OOWS [5], UWE [6], OO-H [7], and Hera [8]. Typically these 
methodologies distinguish themselves from standard information system development 
methodologies by their explicit attention for the navigation design. Since however the 
WIS applications contain content that is highly dynamic, the design has to support the 
dynamics involved with the content. This support includes not just updating the content 
stored in the system, but also allowing the user to affect the hypermedia presentation 
of the data. Illustrative examples of this influence of the user on the hypermedia pre- 
sentation are the history facility that allows the user to go back outside of the presented 
hyperlinks, or the shopping basket concept that allows the user to store some information 
temporarily during a browsing session. Such influence implies that a certain “state" is 
stored by the system to allow the user to interact with the hypermedia presentation and 
particularly with its navigational structure. 

As we indicated earlier the available WIS engineering methodologies have a strong 
focus on the generation of navigation over the content. The user’s actions consisted of 
following links, and as a consequence all the system could do was based on that. The 
history facility is a straightforward example. Giving the user more possibilities to interact 
with the generated hypermedia presentation can help to define or limit the hyperspace 
and thus to realize personalization and adaptivity. In a museum application asking the 
user to define what is interesting for him can help the system to create a more suitable 
navigation structure with specific information about those items on display that interest 
the user/visitor. Another example of user influence would be the role of a shopping basket 
in the sales communication based on a product catalog; not so much for the registration 
of the sales order, but certainly for the adjustment of the presentation in accordance 
with the user input, for example by showing a page with the complete contents of the 
shopping basket (order). As one of the consequences of this extended user influence 
there is a need to deal with navigation data, i.e. data primarily there to support the user in 
influencing (e. g. restricting, selecting) the navigation view over the application domain 
data. In this paper we show how to model this dynamic navigation through Hera models 
that allow the specification of the extended user input, the management of navigation 
information, and the effect of both of them on the hypermedia presentation. For this 
specification Hera uses semantic web languages that are very suitable for modeling of 
semi- structured data and describing their semantics. 

In Section 2 we discuss how related work supports this kind of extended user input 
in relation to hypermedia dynamics. Section 3 highlights the main principles of the Hera 
approach, before we discuss in Section 4 the details of extended user input and dynamics 
in Hera: first we present the input controls, then the navigation data model, its effect 
on the application dynamics, and finally the architecture of the implementation. In the 
conclusion we name the main advantages of this approach compared to other approaches. 
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2 Related Work 

In this section we take a closer look at two well-known representative methodologies 
WebML and OOHDM to see how they support modeling of user input and hypermedia 
(navigation) dynamics. 

In WebML [3] the page content and navigation structure is captured in the (advanced) 
hypertext model using a predefined set of modeling primitives. The infrastructure for 
user input consists of data entry units that have associated with them operation units. 
A data entry unit contains a set of input fields that can be filled by users or can have 
default values. Data entry units have one or more outgoing links that are activated when 
the user fills input fields and submits the information. With a link can be associated 
parameters that transfer the input values to the destination unit(s), for example for further 
processing by an operation unit. There are several predefined operation units, for instance 
for activating external web services or content management operations like creation, 
deletion, and update of entities and relations. The whole library of units is open (new 
units can be defined in XML) and contains a number of data entry units for different kinds 
of user inputs. All contextual information passed between the units by link parameters 
is described in separate XML files. 

The user input is in OOHDM [4] specified by means of interface objects that are 
defined on top of the navigation structure specification. The navigation is described 
using navigation classes derived from concept classes, navigation contexts representing 
collections of navigation objects, and access structures like links, indices, or guided 
tours. The interface objects are instances of interface classes expressed by Abstract Data 
Views (ADV). Every ADV defines a set of events (triggered by users) it can handle via 
methods of navigational classes, and a set of attributes that can be perceived by users. 
The processing of user-triggered events is specified in ADV charts, where the events are 
mapped to messages that are sent to navigation objects and can change their state. 



3 Hera Methodology 

The Hera methodology [8,9] is a model-driven methodology for designing WIS. Be- 
fore we concentrate on user input and hypermedia dynamics in the next section, we 
will briefly describe the main aspects of Hera. In response to a user query a WIS will 
gather (multimedia) data possibly coming from heterogeneous sources and will produce 
a meaningful hypermedia (Web) presentation for the retrieved data. The Hera methodol- 
ogy automates such a process by providing high level abstractions (in terms of models) 
that will drive the (semi-)automatic presentation generation. Moreover, Hera enables the 
presentation adaptation based on user preferences and device capabilities, which means 
that the presentation generation takes into account issues like the platform being used 
(e. g. PC, PDA, WAP phone) [10], 

Based on the principle of separation of concerns and for the sake of interoperability 
several models have been distinguished. Because these models are considered Web 
metadata descriptions that specify different aspects of a WIS, we chose to use the Web 
metadata language, i.e, RDF(S) [11.12], to represent all models and their instances. Our 
choice is also justified by the RDF(S) extensibility and flexibility properties that enabled 
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us to extend the language with model specific primitives to achieve the desired power 
of expression. As RDF(S) doesn’t impose a strict data typing mechanism it proved to be 
very useful in dealing with semistructured (Web) data. 

The Hera toolset implements this methodology by offering software for the automatic 
generation of hypermedia based on the different Hera models. In order to facilitate the 
building (and visualizing) of these models, several Visio solutions were implemented. 
Such solution is composed of a stencil that will display all the model shapes, a drawing 
template, and a load/export feature providing the RDF(S) serialization of Hera models. 
Throughout the paper we use a running example based on the metadata associated to 
about 1000 objects from the Rijksmuseum. Figure 1 depicts (in the CM Builder) a part 
of the CM for our example, while Figure 2 illustrates the corresponding AM. 




Fig. 1 . Conceptual model 



The conceptual model (CM) describes the structure (schema) of the application 
domain data. This structure is described using RDFS in terms of concepts and concept 
relationships. A concept has attributes, i.e. properties that refer to some media instances. 
For concept relationships we define their cardinalities and their inverse relationships. 

The application model (AM) specifies the structure of navigational view over the 
application domain data. This structure is also defined using RDFS, where the hyper- 
media presentation is described in terms of slices and slice relationships (inspired by 
RMM). A slice is a meaningful presentation unit that groups concept attributes (from 
CM) that need to be presented together on the user display. There are two types of slice 
relationships: compositional relationships (for embedding a slice into another slice) and 
navigational relationships (as hyperlink abstractions). 
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Fig. 2. Application model 



4 User Input and Hypermedia Dynamics in Hera 

In most WIS design methodologies, the only kind of interaction considered is link follow- 
ing: the use of the navigation structure is equivalent to wandering through the structure 
by clicking on anchors and following links. In our extended approach we go a step further 
and consider other forms of user input and dynamics with respect to this hyperstructure. 
Therefore, in the next subsections we describe: 

- information for navigation dynamics, defined in the navigation data model 

- user input controls with associated processing of navigation information 

- application model extended with the user input 

- architecture of a Hera system 

We illustrate this by an example from our museum application that allows the visitor 
to buy posters of the paintings in the museum. 



4.1 Navigation Data Model 

In addition to the data in the aforementioned models CM and AM, interaction requires 
a support for creating, storing, and accessing data that emerges while the user interacts 
with the system. This support is provided by means of a so-called navigation data model 
(NDM). The purpose of this model is to complement the CM with a number of auxiliary 
concepts that do not necessarily exist in the CM (although this is the decision of the 
designer in concrete applications) and which can be used in the AM when defining the 
behavior of the application and its navigation structure. 

The NDM of our example is depicted in Figure 3 ; it consists of the following concepts : 
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- The SelectedPainting concept is a subclass of the Painting concept from the 
CM. It represents those paintings which the user selected from the multi-selection 
form. 

- The Order concept models a single ordered item consisting of a selected painting 
(the property includes ) and the quantity represented by an Integer. 

- The Trolley concept represents a shopping cart containing a set of orders linked by 
the property contains. 




Fig. 3. Navigation data model 



We remark that from the system perspective the concepts in the NDM can be divided 
into two groups. The first group essentially represents views over the concepts from the 
CM, the second group corresponds to a locally maintained repository. A concept from 
the first group can be instantiated only with a subset of instances of a concept existing 
in the CM, without the possibility to change the actual content of the data. A concept 
from the second group is populated with instances based on the user’s interaction, i.e. 
the data is created, updated, and potentially deleted on-the-fly. 

The instantiation of both groups of concepts is triggered by a certain action (an 
acknowledgement such as pressing the submit button) specified in the AM. Each such 
action can have an associated query which either defines the view (the first group) or 
specifies what instances should be inserted in the concept’s extent (instantiation). The 
data resulting from the query execution is represented in the NDM instance (NDMI) and 
stored as state information till the next change (query) occurs 1 . The AM can refer to the 
concepts from NDM as if they were representing real data concepts. 

In the example the SelectedPainting concept belongs to the group of view concepts 
whereas both the Order and the Trolley are updatable concepts with the values deter- 
mined at runtime. This is reflected also in the NDMI depicted in Figure 4 that results from 
the user’s desire to buy 3 posters of the selected painting. The instance Paintingl comes 
from the CM, i.e. it is not (re)created: what is created however, is the type property as- 
sociating it with the SelectedPainting concept. Both instances Order 1 and Trolley 1 
are created during the user’s interaction; they, as well as their properties, are depicted 
in bold in Figure 4. Note that for presentation purposes (backwards link generation) we 
also generate for every property its inverse. 

1 We can see an entire spectrum, going from updating the content to just using state data to help 
change the hypermedia structure. In this paper we focus on the state data that helps specifying 
the interaction with the navigation structure (since updating the content is possible but outside 
presentation generation). 
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Fig. 4. Navigation data model instance 




(a) Good input (b) Bad input 

Fig. 5. Form with input in browser 



4.2 Input Controls 

In Figure 5(a) we see from the implementation a slice of a painting selected by the user. It 
shows that in this slice the user is provided with a form to enter a quantity that represents 
the number of posters of this painting that the user considers to buy. In Figure 5(b) we 
see another example where the form is instructed to respond to the user’s attempt to enter 
a non-integer value. 

In the Hera software we implemented the user input forms using the XForms [13] 
standard. As an XForm processor we used formsPlayer [14], a plug-in for Internet Ex- 
plorer 2 . In defining application forms we were inspired by XForms’ clean separation of 
data from controls. 

For these forms we need primitives in the AM that specify the functional embedding 
of the controls in the navigation structure. Figure 6 shows three examples of how we 
specify the embedding of controls in AM. In the leftmost example, the SelectForm 

2 The small logo labelled “fP" in Figure 5(a) is the /ormsPlayer signature in the implementation. 
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allows to make a choice for multiple items out of a list of paintings. The AM primitive 
shows the concept that “owns" the form, in this case Painting ; it shows the items that 
are displayed in the form, in this case names; finally, it shows the items that are handed 
over by the form to the subsequent navigation: the name of the selected painting. In the 
middle example the form is similar but allows a choice of exactly one out of multiple 
options. The rightmost example shows a form called Buy Form that allows user input, 
in this case to enter the quantity of posters considering to buy. The form hands over the 
tuple consisting of the entered quantity and the painting name (the painting information 
is taken from the form’s context). 



Property with range Property with range No input values 

Painting Painter 




submit submit submit 



Multiselection from Selection of one from 

predefined values predefined values 



Fig. 6. Forms in AM 



So we see that for the user input controls we specify in the diagram for the AM the 
relevant parameters that make up the form. Thus we describe the relevant functional 
aspects of the form, and are able to abstract in the diagram from the actual form code. 
Similar to XForms we distinguish between the input controls and their state information 
stored in separate models. 

Figure 7 presents the models for the forms SelectForm. and Buy Form. It consists 
of two form types, Forml defines the type of the SelectForm and Form2 defines the 
type of the BuyForm. A Hera form model instance represented in RDF/XML corre- 
sponds to the associated XForms model instance. The Integer type matches the XML 
Schema [15,16] type xsddnteger and the String type matches the XML Schema type 
xsd'.string. In case that the user enters a value of a different type than the one specified 
in the form model, an XForms implementation (see Figure 5(b)) will immediately react 
with an error message (due to its strong type enforcement capabilities). 




Fig. 7. Form models 





68 



G.-J. Houben et al. 




Fig. 8. Form model instances 






Fig. 9. Extended application model 



Figure 8 describes two possible model instances for the form models given in Fig- 
ure 7. In the SelectForm the user selected two paintings and in the Buy Form the user 
decided to buy one of these paintings. 

4.3 Application Model 

With the aid of the aforementioned primitives we are able to express the user input in 
our museum example in terms of an (extended) AM. Figure 9 depicts the part of the AM 
which captures the user input. 

The Technique slice contains a form that lists all paintings exemplifying that partic- 
ular technique and offers the user the possibility to select some of these paintings. For the 
latter we see in the T echnique slice the input control called SelectF orm with Painting 
as its owning concept (meaning that this form is selecting Painting concepts). We also 
see that the form lists the paintings by their aname property and produces for each 
selected painting the aname property to identify the selected paintings. 














Modeling User Input and Hypermedia Dynamics in Hera 



69 



After selecting a painting, the outgoing slice navigational relationship denotes that 
the form in the T echnique slice results in navigation to a slice that represents the set 
of selected slices, each represented by their aname, and also in a Trolley that, while 
initially empty, will contain the paintings that actually are going to be bought. A Trolley 
contains a set of Orders, while an Order represents the request to buy a poster of a 
(selected) painting in a certain quantity. 

The navigation can go further to the SelectedPainting slice. That slice includes 
not only all the properties that represent the painting, but also a form called Buy Form. 
with a user input control. That control allows the user to specify the quantity (of posters 
of this painting to buy). After filling this form the user can navigate via the outgoing 
slice relationship to the next slice where the trolley is maintained (and where the user 
can decide to select another painting for considering in more detail). 

With these slices and slice navigational relationships in the AM we have specified 
the entire navigation structure. In the AM diagram we exploit the fact that the function- 
ality of the controls is standard, e. g. the selection of n items from a list; therefore the 
diagram only indicates which standard control is used. What we also do indicate is the 
signature: we give the properties displayed in the form, and the identification of the con- 
cepts forwarded via the slice navigational relationship. Note that both slice navigational 
relationships that emerge from the forms (Q 1 and Q 2) are in fact queries. In the query 
definition we will use the prefix cm : for concepts/properties coming from the conceptual 
model, the prefix ndm : for concepts/properties specified in the navigation data model, 
and form: for concepts/properties introduced in the form model. 

The RDF model instance of SelectForm is given in Figure 10. The query Q 1 creates 
a view over painting instances from the CM instance (CMI) which were selected by the 
user in SelectForm. This view defines the instances of the ndm: SelectedPainting 
class from the NDM. 



<Forml rdf : ID="SelectForm"> 

<aname>The Stone Bridge</aname> 
<aname>Portrait of Maria Trip</aname> 
</Forml> 



Fig. 10. Model instance for SelectForm 



Figure 1 1 describes this query in the SeRQL [17] notation. The actual form is mod- 
elled as an RDF resource with multiple f orm:aname properties containing the names 
(values) of those paintings which were selected by the user. 

The RDF model instance of BuyForm is given in Figure 12. The query 02 associ- 
ated with the BuyForm creates a new instance of the NDM concept ndm:Order each 
time the user decides to buy a poster of a selected painting. 

The SeRQL translation of this query is presented in Figure 13. The form is mod- 
eled similarly as before by an RDF resource with two properties f or m: quantity and 
f ornv.aname. Note that the (old) instance of the ndnv.Trolley exists outside this form 
and is created beforehand during the initialization of the session. 
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CONSTRUCT 

{P}<rdf : type>{<ndm : Select edPaint ing>} 

FROM 

{P}<rdf : type>{<cm : Painting>} ; 

<cm : aname>{Paname} 

WHERE 

Paname IN SELECT Faname 

FROM {SF}<f orm: aname>{Faname} , 

{SF}<rdf :type>{<form:Forml>}, 
{SF}<rdf : ID>{Fname> 

WHERE Fname = "SelectForm" 



Fig. 11 . User query Q1 



<Form2 rdf : ID="BuyForm"> 

<aname>The Stone Bridge</aname> 
<quant ity>3</ quant i ty> 

</Form2> 



Fig. 12. Model instance for BuyForm 



CONSTRUCT 

{0}<rdf : type>{<ndm : Order>} ; 

<ndm : quant ity>{Fquantity} ; 

<ndm : includes>{Fpaint ing} , 

{T}<ndm : contains>{0} 

FROM 

{T}<rdf : type>{<ndm : Trolley>} , 

{Fpaint ing}<cm : aname> {Paname} ; 

<rdf : type>{<ndm : SelectedPaint ing>} , 
{BF}<f orm : quant ity>{Fquantity} ; 

<f orm: aname> {Faname} , 

{BF}<rdf : type>{<f orm : Form2>} , 

{BF}<rdf : ID>{Fname} 

WHERE 

Paname = Faname AND 
Fname = "BuyForm" 



Fig. 13. User query Q2 



4.4 Architecture 

While in the previous subsections we have paid attention to the specification of user 
input and hypermedia dynamics and the way in which the AM can support this extended 
interaction specification, we now turn to the implementation. As we have indicated earlier 
the software of the Hera toolset can generate the hypermedia structure from the given 
models. In other work [8] we have sketched the (software) architecture for the case of 
normal link following. In Figure 14 we see how we extended the architecture such that 
it supports the handling of user input, e. g. via the forms 3 . 

We see that the presentation engine is responsible for serving the generated pre- 
sentation to the user. As soon as the engine discovers user input it hands this over to 
the form processor that is going to interpret the actual user input (and possibly the 
contextual information that explains with what the user input is associated). So, the 
form processor can get the quantity of posters to buy and the context that explains 

3 We have focussed here on the part of the architecture for extended user interaction, and left out 
the architecture description for the rest of the software. 
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conceptual model 




application model 


vocabulary 




vocabulary 


(rdfs) 




(rdfs) 




□O Application independent 
□o Application dependent 
| (^) Query/user dependent 



Fig. 14. Presentation generation 



what the painting is for which the user wants to buy this quantity. The form processor 
can produce data that is added to the NDMI. The information from NDMI and CMI is 
used together in order to generate/update the AM instance. 

As we have indicated in Figure 14 the implementation fits nicely in the RDF-based 
approach that we already had for the link following. By adding the additional model in- 
formation in RDF(S) we can perfectly manage the additional functionality of explicit user 
input resulting in a different hypermedia presentation. We implemented data transfor- 
mations (see cmi2ami and ami2impl in the figure) by means of XSLT stylesheets [18]. 
This was made possible due to the XML serialization of RDF model instances and the 
fact that XForms [13] is XML-based. Besides its XML interoperability XForms offers 
also device independence, which enables to use the same form on multiple platforms. 
X-Smiles [19] provides a good view on how the same XForms will look like on desktop, 
PDA, WAP phone etc. As an XSLT processor we used Saxon [20] which implements 
XSLT 2.0 and XPath 2.0. In the form processor we employed the Java-based Sesame [21] 
implementation of the SeRQL query language [17]. 



5 Conclusion 

By providing new primitives in Hera, e. g. for capturing the user input, it is possible to 
considerably extend the class of applications that can be specified. As an example we 
mention the use of primitives like the “shopping basket" or “list selection" that are so 
typical for applications in the context of services provided via the Web. Another extension 
is the increased support of dynamics. With the new primitives and the navigation data 
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model it has become possible to handle effectively the dynamic adaptivity known from 
adaptive hypermedia [22] . For example, for such personalization purposes the navigation 
data model can store the necessary user model (both its temporary and persistent parts), 
while the application model can specify the necessary adaptation rules. The construction 
of the navigation view over data can be further enhanced by existing generic methods 
for the development of navigation structures based on user interaction modeling, for 
instance [23]. 

Comparing these new facilities in Flera to other work, we see that the explicit support 
of input controls such as forms are not modelled explicitly in OOHDM for example. 
It does distinguish mouse-related aspects of user input, but compared to Hera it does 
have a more limited support of data-entry by the user. This aspect of user input is a 
strong point of WebML, but in fact that facility is more concerned with the content 
management of the information system. In Hera this is also supported, at the level of 
the conceptual model, but besides of this Hera allows to combine the stored data in the 
conceptual model with the auxiliary navigation data stored in the navigation data model. 
WebML has, next to the link parameters, also global parameters that can model a “state" 
in terms of attribute-value pairs, but Hera goes much further in the specification of this 
state information allowing also complex relationships between concepts (represented in 
graphs). 

The fact that Hera uses RDF(S) representations of the models gives a number of 
advantages over other approaches. To start with, it supports the semistructured data that 
is so typical for the Web. With RDF(S) Hera offers increased interoperability, e. g. for 
the exchange and sharing of user models. It also allows to express complex queries (e. g. 
in the design of the dynamics) that make use of the subclassing mechanism. Moreover, 
the modeling in Hera of the extended user input inherits good principles from existing 
standard like XForms. It chooses to separate in the user input the controls (the presen- 
tation aspects) from their models (the data aspects). Opposed to other approaches, Hera 
has a concrete implementation based on these standards. 

In future we plan to incorporate the possibility of web service invocation within 
applications designed using Hera on different levels: on the conceptual (data) level (web 
services will act as virtual instances of data concepts), and on the application level (web 
services will provide building blocks of slices). Furthermore, we investigate general 
properties, mutual relationships, and constraints of Hera models that would help us to 
build tools for the automated checking of correctness of the models. 
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Abstract. OOHDM models hypermedia-based Web applications by an object 
model on three layers. Recently, an OOHDM extension by business processes 
has been proposed. In all cases, the definition includes a formal description of 
the syntactical aspects and a verbal description of the semantics. In this paper, 
we give a behavioral definition of the semantics of the OOHDM core features: 
navigation and advanced navigation; and of the proposed extension by business 
processes. We derive application-specific model classes from predefined 
behavioral model classes that have operations with a well-defined semantics. 
The behavioral model classes collaborate with a Web Application virtual 
Machine (WAM). The WAM models basic Web-browser characteristics, i.e. 
HTTP-HTML characteristics. Thus, the semantics of an OOHDM Web 
application model is precisely defined in an executable way. 



1 Introduction 

The Object-Oriented Hypermedia Design Method OOHDM by Schwabe and Rossi 
[SR98] is a modeling and design method for Web applications, which describes 
hypermedia-based navigation by an object model on three levels, the conceptual 
level, the navigational level and the interface level. Recently, Schmid and Rossi 
proposed an extension of OOHDM by business processes [SR02] [SR04], The 
definition of OOHDM includes a formal description of syntactical aspects and a 
verbal description of the semantics. 

However, verbal descriptions like that are not always precise, but often vague and 
open to misunderstandings and doubts. A formal definition of the semantics is 
required to cope with this problem. Since OOHDM uses object models, it lends itself 
to a behavioral definition of the semantics in form of the object behavior. 

We focus in this paper on a behavioral definition of the semantics of the OOHDM 
core features, which are navigation and advanced navigation; and of the proposed 
OOHDM business process extension. OOHDM models a Web application by 
application-specific classes with a semantics that is given verbally. Our approach is to 
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derive these classes from predefined behavioral model classes that have operations 
with an executable semantics definition. The behavioral model classes, called shortly 
model classes, collaborate with a Web Application virtual Machine (WAM). The 
WAM models basic Web-browser characteristics, i.e. HTTP-HTML characteristics. 
Thus, the semantics of an OOHDM model of a Web application is precisely defined 
in an executable way. 

We present and explain an OOHDM model of a Web shop that includes 
navigation, advanced navigation and a business process in section 2. Section 3 
introduces the Web Application virtual Machine WAM and related classes and 
services. Section 4, 5 and 6 define the semantics of the OOHDM navigation, 
advanced navigation and business process constructs by a behavioral model. Section 
7 surveys shortly related work. 



2 The Web Shop as an Example for an OOHDM Model 

We use the Web shop presented in [SR04] as an example for an OOHDM model that 
includes navigation, advanced navigation and a business process. OOHDM (for 
details see [SR98]) models the objects forming the application domain in a conceptual 
schema (see Figure 1); it models abstracted Web pages and the navigation 
possibilities among them in a navigational schema (see Figure 2); and the 
presentation aspects of Web pages (which we disregard) in an interface schema. 

The conceptual schema is partitioned in entities (bottom) and in processes with 
classes and activities (top), and the navigational schema in entity nodes, among which 
you may navigate (left), and in activity nodes that belong to processes (right). UML 
stereotypes, which OOHDM considers just as a classification, indicate the category to 
which each object class belongs. But for the behavioral model definition, each 
stereotype indicates the model class an application-specific class is derived from. 
Note that the classes shown in Figure 1 and 2 do not give a complete application 
model of the example Web shop, to avoid an overloading with details. The required 
details will be given in sections 4-6. 

The relationship between objects in the conceptual and navigational schema, like 
that between an entity node CDNode and an entity CD, is given explicitly in the 
OOHDM node definition syntax [SR98], but represented in the schemas only by the 
correspondence of the names. 

Pure Navigation 

Consider, for example, the navigation possibilities in the CD store. On the left of the 
navigational schema (see Figure 2), you find entity nodes (which are an abstraction of 
Web pages) like CDNode or ShoppingCartNode. Links among nodes are represented 
as directed edges which may be labeled with the link name. There are links 
representing the navigation possibilities from the customer HomePageNode to the 
CDNode or to the ShoppingCartNode, from the CDNode to the ShoppingCartNode 
and back, and from one CD to other related CDs by using the “related” link between 
CDNode ’s. 
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Advanced Navigation 

More advanced hypermedia applications are not composed of read-only pages; they 
use Web pages as the interface for triggering different kind of actions that may 
change the internal state of the application. An atomic action, like adding a product to 
a shopping cart, calls an operation of an entity like ShoppingCart. When the user 
presses the “add to shopping cart” button on the Web page that displays the CDNode, 
called CDNode interface on the OOHDM interface layer, that button invokes, as 
described on the OOHDM interface layer, the addToCart operation of the selected 
CDNode. The CDNode sends the message add( CD) to the ShoppingCart object, 
which changes its state. 




Fig. 1 . OOHDM conceptual schema of a Web shop including entities and a business process 

Business Processes 

An entity object like CD or Customer has a permanent lifetime and state; a process 
object like Checkout has a temporary state and no permanent lifetime (see [S99] 
[SR04]). Processes and activities are modeled in the OOHDM conceptual schema (see 
Figure 1 top), and activity nodes in the OOHDM navigational schema (see Figure 2 
right). 

Typically, a business process like Checkout (see Figure 1 top) is composed of 
several activities like Login, Confirmltems, SelectShippingAddress, etc. This is 
represented by an aggregation relationship in the conceptual schema. We consider a 
business process as a root activity that may consist itself of a set of activities. An 
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activity is either basic, like Login etc., or composed from other activities, like 
Checkout, following the composite pattern [GHJV95]. An activity collaborates with 
application entities, like Login with Customer. An activity provides operations like 
start, next, commit, suspend and resume, that allow to execute a business process. 



«activity node container» 

CheckOutNodeContainer 



«activity node» 

SelectDeliveryOptionsNode 



+next() 



next 



± 

«activity node» 

SelectPaymentOptionsNode 



+commit() 



addToCart() { 
value = this.getField(key); 
shoppingCart.add(value); } 

terminate 







«dyn entity node» 

ShoppingCartNodi 



+startCheckout() 



«fixed entity node) 

HomePageNode 



t; 






~1K~ 



-A 



«dyn entity nodes 

CDNode 



+addToCart() 

+resumeCheckout() 



"7fC 



^uspend 



«activity node» 

LoginNode 






+next() 








«activity node» 

ConfirmltemsNode ^ 



+suspendCheckout() 
+next() 



«activity nodes >, 

SelectShippingAddressNode \ 



Fig. 2. OOHDM navigational schema of a Web shop including navigation, advanced 
navigation and a business process 

An activity node like LoginNode models abstractly a Web page, presenting the 
output of an activity and accepting its input. The OOHDM interface layer (not shown) 
describes which buttons, like commit or next, a node contains. Pressing a button 
triggers a matching method of the activity node, like next of LoginNode, which calls 
the matching method of the activity, like «ejtt(d:LoginData) of Login, passing the user 
input. An activity node is shown in the context that is created by its process. This 
context is represented as an ActivityNodeContainer, like CheckOutNodeContainer 
(Figure 2 right). 

An edge labeled with a reserved label start, terminate, suspend, resume or next 
does not represent a navigational link, but the possibility to go from or to get to an 
activity node by process execution. Informally, the edge semantics is as follows (see 
[SR04]). The start edge from the ShoppingCartNode to the CheckOutNodeContainer 
represents starting the Checkout process from navigation, the terminate edge 
terminating it and taking navigation up again. A next edge among activity nodes, e.g. 
from LoginNode to ConfirmltemsNode, represents the transition from one to the next 
activity, including the completion of the first and the starting of the next activity. An 
activity diagram (not shown) that forms part of the conceptual schema shows the 
possible flow of control among the activities. 

A Web application may allow that a business process is suspended to do 
temporarily some navigation. E.g., a user may suspend checking out at the 
ConfirmltemsNode and navigate to the CDNode to get more information about CDs 
he is buying. This is represented by a suspend edge. A resume edge from an entity 
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node to an activity node container, like the one leading from CDNode to 
CheckoutNodeContainer, represents returning from the temporary navigation and 
resuming the business process at the state it was suspended. 



3 Model Classes and the Web Application Virtual Machine 

To define the semantics of application-specific classes like those presented in Figure 
1 and 2, we derive them from behavioral model classes with a predefined semantics. 
We use the UML stereotype “model" to label a model class. The basic model classes 
are: Entity, RootActivity, and BasicActivity on the conceptual layer; Node and Link 
on the navigation layer; and InteractionElement on the interface layer. Node, Link 
and InteractionElement are specialized as described in the next paragraphs and Figure 
3. The model classes collaborate with the Web Application virtual Machine, 
abbreviated WAM. The WAM models basic Web-browser characteristics, i.e. HTTP- 
HTML characteristics as seen from a Web application. 




Fig. 3. Specialized behavioral model classes Node. InteractionElement and Link 



The model class Node models a Web page abstracting from many of its concrete 
characteristics, and the model class InteractionElement models interaction elements 
like anchors and buttons. 

The class InteractionElement defines the operation clickedQ which is called when 
an end user clicks at the interaction element. We refine the class InteractionElement 
in Anchor and Button, and Anchor in FixedPageAnchor and DynPageAnchor. Their 
detailed characteristics are presented in section 4 and 5. 

The class Node defines the (abstract) operations getPage{)\ WebPage, getField{ 
n:Name): Value, setField(w. Name, v: Value), getFieldNcimesQ: Name []. It contains 
an array of InteractionElements. We refine a Node into EntityNode and 
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ActivityNode: an EntityNode may contain Anchor’s and Button’s, an ActivityNode 
only Button’s as InteractionElements. The class Link is refined into FixedPageLink 
with a method traverse (), and DynPageLink with a method traverse^ k: Key). 

The WAM has the attribute currentNode, a reference to the currently displayed 
Node, and an operation display^ n:Node) {currentNode=n; showPage(n.getP<7ge());}. 
When a user enters or edits data on the currently displayed page, the WAM calls the 
setField-method of the currentNode to change the state of the Node, When a user 
clicks at an interaction element, the WAM calls the operation clicked of the 
corresponding Anchor or Button of the currentNode, 

In the following sections, the behavioral model gives the executable definition of 
the operations in the form of UML notes in Java. As an alternative to Java, we might 
use the action semantics of UML 1. 5/2.0. Operations defined by the behavioral model 
are printed in bold fonts to distinguish them from application-specific operations. For 
lack of space, the behavioral model presented does not include items like error cases. 



4 The Behavioral Semantics of Navigation 

Navigation allows a user to navigate from a given node to a linked node by clicking 
at the link. Its main characteristic is that the state of navigation is determined only by 
the page displayed by the browser and its content [SR04], The behavioral model of 
navigation mirrors this characteristic: it shows that navigation does not cause any 
state changes but that of the current node of the WAM. 

We distinguish two kinds of navigation, navigation among Web pages with a fixed 
content and navigation among Web pages with a dynamic content. 

Navigation Among Web Pages with a Fixed Content 

Fixed page navigation follows a link to a node without dynamically generated 
content, like the link from CDNode to HomePageNode in the navigational schema 
Figure 2. The behavioral model shown in Figure 4 contains, besides the user-defined 
classes for the source and target node of the link, only the model classes: Anchor, 
FixedPageAnchor, and FixedPageLink; which are instantiated and configured to work 
together. These model classes have executable definitions of the methods clicked, 
navigate and traverse. 

A node like the CDNode contains a FixedPageAnchor instance, which references a 
FixedPageLink instance, which, in turn, references the target node of the link. Each 
reference is set by a constructor parameter. When the WAM displays a Web page, i.e. 
a Node, and a user clicks at the Anchor of a fixed page link, the WAM calls the 
(7hTec/-operation. This forwards the call to the navigate operation of the 
FixedPageAnchor, which calls the traverse- operation of the FixedPageLink. The 
traverse- operation calls the display-operation of the WAM with the target node as a 
parameter. The WAM sets, as described in section 3, that node as the current node 
and calls its getPage- operation in order to display the page. 
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Fig. 4. Static navigation from CDNode to HomePageNode 
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Fig. 5. Dynamic navigation from ShoppingCart Node to CDNode 



Navigation to Web Pages with a Dynamically Generated Content 

Navigation to a dynamic page is similar to fixed page navigation, except for the key 
required to identify the dynamic content (see Figure 5). The differences are: 

- A source node of a dynamic link has an operation getLinkKey that returns a key 
identifying the dynamic content of the target node. 

- A DynEntityNode defines an operation find( k: Key) that fetches the dynamic 
node content from the associated entity (calling its static /i«r/( k: Kcy)-operation), and 
a vet-operation to set the (found) dynamic content into the attribute fields. 



- The class DynPageAnchor has a navigate - operation that fetches the key of the 
dynamic content from the source node, called myNode, and passes it as a parameter 
with the call of the traverse- operation. The traverse - method of the DynPageLink 
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class calls the/Zncf-operation of its target node of type DynEntityNode with the key as 
a parameter, and then its vet-operation so that the target node sets its dynamically 
generated content. Last, it calls the display- operation of the WAM with the target 
node as a parameter. 



5 The Behavioral Semantics of Advanced Navigation 

Advanced navigation allows a user to trigger an atomic action by pressing a button on 
a Web page. An atomic action enters or edits information in a Web application, 
modifying the state of application objects that are modeled in the conceptual schema. 
Consequently, the Web application state is composed by the state of the current node 
displayed by the browser, and by the state of the application objects. The behavioral 
model of advanced navigation shows that not only the current node of the WAM, but 
also application domain objects may change their state. 

For example, consider the addToCart operation of the CDNode in Figure 2, which 
is triggered by the AddToCartButton. The behavioral model for advanced navigation 
(see Figure 6) shows the model class Button with the operations clicked and action. A 
derived application-specific class, like AddToCartButton, implements the action- 
method. which calls an operation of the source node, like addToCart of CDNode. 




Fig. 6. Triggering the execution of the atomic addToCart-nctwn by a button 



When the WAM displays a Web page, i.e. a node, and a user clicks at a button on 
this page, the WAM calls the operation clicked of this button. The clicked-method 
forwards the call to the action- method, which forwards the call to the addToCart- 
method of the CDNode. This method fetches the value from the key-field of the 
(currently presented) CD and sends the message flrW(value) to the ShoppingCart 
object, which changes the state of the shopping cart. 

Note that the execution of an atomic action does not imply the navigation to 
another node. This would have to be modeled by adding a Link to CDNode and 
addToCart calling additionally the traverse-method of the link. 
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6 The Behavioral Semantics of Business Processes 

The main characteristics of a business process (see [SR04]) are that it: 

• drives the user through its activities. This means it defines the set of activities to 
be executed, and the possible control flow among them. 

• keeps its state internally. The state can be changed only by the process itself on a 
user interaction, but not by the user pressing a browser button. 

The behavioral semantics of a business process mirrors these characteristics. The 
behavioral model defines the semantics of process state transitions, which are 
represented by process-related edges like start and next in the navigational schema 
(see Figure 2): it shows that a business process changes its state only when a process 
state transition is triggered by the user, and that, in general, the process, and not the 
user, selects the next activity for execution. 

To make the behavioral model not unnecessarily difficult to understand, we make 
the restriction (not made by [SR04]) that there are only two levels of process 
execution, i.e. a process consists of a root activity with basic activities as children. 
The behavioral model defines the classes RootActivity and BasicActivity, with 
operations like start, run, next, commit, suspend, and resume with a predefined 
semantics, given by final methods or inherited from class Thread. Application- 
specific activity classes are derived from them. 

A business process is executed independently from navigation in an own thread, so 
that it can preserve its state when suspended, and be resumed at the point of the 
suspension. We use Java threads to model the business process threads and navigation 
thread. The model class RootActivity, derived from Thread, inherits Thread- 
operations to start, suspend, resume and complete a business process thread. 



6.1 Starting and Terminating a Business Process and an Activity 

A business process is started from an action method of an entity node. E.g., the 
startCheckout- method of the ShoppingCartNode (see Figure 2 and 7), triggered by a 
user pressing a StartButton, starts the Checkout process and suspends the navigation 
thread, calling suspend of the Thread under which it runs. The mechanics of buttons 
and invocation of the action- method is exactly the same as described in section 5; so 
we do not present it again here and in the following examples and figures. 

The behavioral model for starting and terminating a business process is shown in 
Figure 7. The start-method that Checkout inherits from Thread calls the predefined 
run-operation of the business process thread. The final run-method of the model class 
RootActivity, which realizes that operation, is a template method [GHJV95] that is 
inherited by the application-specific Checkout root activity. It calls the application- 
specific methods startHook for initialization and getN ext Activity to select the basic 
activity to be executed, sets this activity, like Login, as current (-ly executed) activity, 
and starts it. 
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Starting a Basic Activity 

The predefined vfa/T-operation of the model class BasicActivity initializes the content 
data of the associated activity node and displays it as Web page. It is a template 
method that is inherited by an application-specific activity like Login. The start- 
method calls the application-specific startHook-method. This method usually gets the 
data to be presented to the user from the application domain objects and puts them 
into the attribute fields of the associated ActivityNode; but the Login activity presents 
no data in the LoginNode. Then the .v/wf-method calls the display- method of the 
WAM passing the activity node like LoginNode as a parameter, so that the WAM 
displays it as a Web page. 

Completing a Basic Activity 

All activity nodes of the Checkout process have a next-button to complete the activity 
and go to the next one, except for the SelectPaymentOptionsNode which has a 
commit-button, since it completes the SelectPaymentOptions activity and then the 
Checkout process. 
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Fig. 7. Starting the Checkout process from the ShoppingCartNode and completing it from the 
SelectPaymentOptionsNode. 

For example, the LoginNode has a next button which triggers its next- method. This 
method fetches the data from the LoginNode which the user has entered during the 
Login activity, and passes them as parameters with the call of the method next{ 
d:LoginData) of the Login activity. This method compares the entered user name and 
password with the associated Customer object, and calls, if the check is successful, 
the next-method of Checkout. This method is a template method inherited from the 
model class RootActivity; it selects the activity to be executed next by calling the 
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application-specific hook method getN ext Activity, sets this activity as current (-ly 
executed) activity and starts it. 

Terminating the Checkout Business Process 

A business process like Checkout is terminated when it has completed its work, that is 
when the last of its child activities, SelectPaymentOptions, has completed its work 
and Checkout has completed the customer order and sent it off. 

The SelectPaymentOptions activity completes its work when the commit-button 
triggers the commit- method of the SelectPaymentOptionsNode (see Figure 7 bottom 
right). This method fetches the payment options data, which the user has entered 
during the SelectPaymentOptions activity, and passes them as parameters with the 
call of the commit( d: PaymentData) method of the SelectPaymentOptions activity. 
This method checks the entered data and stores them in the PaymentOptions object. 
Then it calls the commit- method of its parent activity Checkout. 

The Checkout activity inherits the template method commit from the model class 
RootActivity. The commit- method calls the application-specific commitHook to 
complete the processing of the order before it resumes the navigation thread (with the 
Thread resume- operation) and calls the application-specific traverseCommit Link- 
method. This method calls the traverse- operation of the FixedPageLink to the 
HomePageNode, so that the WAM presents the home page to the user. The business 
process thread is completed with the end of the commit- method. 



6.2 Suspending and Resuming a Business Process 

A business process like Checkout may be suspended only from activity nodes where 
this is provided for by a Web application designer. E.g., the ConfirmltemsNode has a 
method suspendCheckout that is triggered by a user pressing a NavigateToCD-button. 
The behavioral model for suspending and resuming a business process is given in 
Figure 8. 

The suspendCheckout-method of ConfirmltemsNode calls the resume- operation of 
the navigation thread, and the traverse-method of the suspendLink in order to 
navigate to the CDNode, before it calls suspend of Confirmltems to suspend the 
business process. 

The Confirmltems activity inherits the suspend- method, which is a template 
method, from the model class BasicActivity. This method calls the suspendHook, 
which saves the activity state if required, as e.g. after user inputs during the activity, 
and stores it temporarily in the activity state, and then the suspendBP-method of the 
Checkout activity. The suspendBP-method, which the Checkout activity inherits from 
the model class RootActivity, calls the application-specific suspendHook that 
performs application-specific actions before the suspension, and then the Thread 
suspend- operation to suspend the business process thread. 

Resuming the Checkout Business Process 

A business process like Checkout may be resumed only from entity nodes where this 
is provided for by a Web application designer. E.g., CDNode has a resumeCheckout- 
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method that is triggered by a user pressing a ResumeCheckout-Button during the 
temporary navigation. Figure 8 gives the behavioral model for resuming a business 
process, which suspends the navigation thread. 

The resumeCheckout-method of CDNode calls resume of the Checkout process (if 
Checkout is not suspended, an exception is returned) and then the ra.s/?e«£/-method 
that the navigation thread inherits from Thread. The resume- method, which Checkout 
has inherited via the model class RootActivity from Thread, resumes the business 
process thread which has been suspended in the suspendBP- method. When this 
method is resumed, it calls the application-specific resumeHook , which may e.g. 
restore some state if required. Finally, it calls the resume- method of the basic activity 
like Confirmltems that was active when the business process was suspended. 
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- ConfirmltemsNode 




+myActivity : Confirmltems 
+suspendLink : DynPageLinl 


+suspendCheckout() 
+getCDKey() : Key 



resumeCheckout() { 

checkout. resume(); //Thread 
suspendQ; //this Thread } 






1 



«model» 

DynPageLink 



+traverse(k:Key! 






«entity node» 

CDNode 



-checkout : Checkout 



+resumeCheckout() 



Fig. 8. Suspending the Checkout process from ConfirmltemsNode and resuming it from 
CDNode 

Confirmltems has inherited the resume- method from the BasicActivity class. The 
resume- method calls first the application-specific resumeHook. This method restores 
the data, which were presented to the user and saved when the activity was 
suspended, from the application state or temporary objects, and puts them into the 
attribute fields of the associated ActivityNode, like ConfirmltemsNode. The resume- 
method calls then the display- method of the WAM passing the ConfirmltemsNode as 
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a parameter so that the WAM displays the ConfirmltemsNode as a Web page, and the 
user can go on with the execution of the Checkout process. 



7 Related Work 

Many Web application design methods proposed in the last years, like WebML by 
Ceri, Fraternali, and Paraboschi [COO] and W2000 by Baresi, Garzotto, and Paolini 
[BOO], do not have special constructs to model business processes. Recent proposals 
like UWE by Koch and Kraus [KKCM03], OO-H by Cachero and Melia [KKCM03], 
and OOWS by Pastor, Fons and Pelechano [PFP03] have constructs for the modeling 
of business processes., but do not give a formal definition of the semantics of the 
model constructs. OOWS captures functional system requirements formally to 
construct from them the Web application. 



8 Conclusions 

We have given a behavioral definition of the semantics of OOHDM and of its 
business process extension. The definition is not complex and easy to understand. 
One reason for that seems to be that the OOHDM designers did an excellent work: the 
OOHDM conceptual and navigational layers, and a few aspects of the interface layer, 
seem to focus exactly on all important aspects of a Web applications, without giving 
unnecessary details nor a description of platform- or technology dependent aspects. 
Another reason seems to be that OOHDM as an object-oriented approach lends itself 
to a behavioral definition of the semantics. 

We have defined a Web Application virtual Machine WAM. Application-specific 
classes and behavioral model classes collaborate with it. A version of the WAM that 
neglects the Web page layout given by the OOHDM interface schema, is easy to 
implement. With that WAM version, both the behavioral definition of the OOHDM 
semantics and an OOHDM model of a Web application can be directly executed and 
tested. 

Another very promising aspect is the use of the behavioral OOHDM semantics 
definition for a model driven architecture approach. For example, the entities from 
the conceptual layer may either model e.g. Corba or Enterprise JavaBean application 
objects from an existing back-end application, or they may conceptualize non- 
existing lightweight entities that provide just an interface to a database management 
system. In the first case, a generator just generates code to invoke the existing 
application objects from the servlets that realize the OOHDM nodes, whereas in the 
second case, a generator generates servlets that realize the OOHDM nodes and 
embody the database access provided by the lightweight entities. 
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Abstract. XGuide introduces the concept of contracts to the Web engineering 
domain. Contracts define the characteristics of Web components and pages. They 
serve as specifications for pages, enforce strict separation of concerns, are compos- 
able and enable concurrent implementation. As a result, time-to-market is short- 
ened and support for maintenance and evolution scenarios is provided. The XGuide 
development process iteratively extends and refines conceptual models towards 
concrete implementations and thus bridges the gap between the conceptual and 
implementation space. 



1 Introduction 

Various methods were proposed to support Web engineering [1] processes — often in- 
fluenced by and building on results from other fields of computer science. Existing ap- 
proaches include database-centric, graph-based, logic-based, object-oriented and com- 
ponent-based methods. Most approaches share several key requirements and concepts. 
The ability to model a Web-based system on a conceptual level that is subsequently 
transformed into a concrete implementation is one. Dividing the system into smaller 
modules and components that can be reused and composed to form larger components 
is another. Additionally, the principle of separation-of-concems is identified as being 
crucial and is supported to various degrees. 

The goal of the XGuide approach is to propose precise specifications and design- 
by-contract [2] for Web pages and Web components (page fragments). We call these 
specifications Web contracts. Web contracts serve as the basis for the composition and 
integration of Web components into Web pages. Furthermore, they support the separation 
of design concerns by clearly specifying the concerns (and their dependencies) involved 
in the creation of a page or component. Such concerns include the content management, 
layout design and application logic programming. After the contract design, XGuide 
includes a step-wise refinement of these design artifacts towards a concrete implemen- 
tation. Based on the concern and dependency specifications in the contracts, XGuide 
strives to reduce the overall time-to-market by supporting a concurrent implementation 
phase. A prototype development environment called XSuite supports and automates the 
XGuide method. 

2 Contract-Based Development with XGuide 

The XGuide development process is structured into seven phases: requirements analysis, 
feasibility decision, conceptual modeling and design, implementation, testing, deploy- 
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ment, and evolution and maintenance. For space reasons, we focus on the design phase 
in this paper. A complete discussion of the XGuide method can be found in [3]. 

In the design phase, a conceptual model of the Web site is created. This model 
consists of pages, components, component references and application logic processes. 
To illustrate the approach, we use a snippet from the Vienna International Festival (VIF, 
http: //www. f estwochen . at) 2003 case study. The VIF is an annual cultural 
festival in Vienna featuring operas, theatre performances, concerts, exhibition and other 
cultural events. The VIF Web site not only offers information about the programme, 
venues, press releases, etc. but also provides a shopping cart application for ordering 
tickets online. Figure 1 displays a simplified version of the conceptual model for the 
shopping cart of the VIF Web site. 




Fig. 1 . A snippet from the component web for the VIF shopping cart application. 



A page (depicted as a rectangle in the model) denotes a unit that will be presented 
to the user in her browser. From the user’s point of view, all interactions are with pages; 
the internal structure of pages (i.e., its decomposition in components) is not visible. 
Components (depicted as rectangles with curved lower borders) represent page fragments 
that are reused on several pages. Typical examples for components are page headers, 
footers and global access structures such as hierarchical navigation menus. A component 
can be included in another component or page by referencing it in the References section 
of the component or page. For the purpose of this discussion, we only defined a single 
component Header in Figure 1 that is included in all pages. Finally, application logic 
processes (rectangles with rounded corners) represent the back-end processing (if any) 
that takes place on a server round-trip (e.g., when moving from one page to another by 
following a link or submitting a form). 

So far, we did not distinguish between static and dynamically-generated pages or 
any information flow between them. The content of dynamic pages usually depends on 
some input values provided by the framework or the user. Customized navigation bars, 
for example, take a page identifier as input and render the currently viewed entry in the 
menu differently than the others. In our example, the ShowDates page requires an event 
identifier as input to determine which event’s dates it should present. In XGuide, we 
denote the set of arguments a page requires to be created at the server side as the page’s 
input intetface. 
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After the input interfaces for all components and pages, we can define the corre- 
sponding output interfaces. An output interface is the opposite concept and describes 
what values a page, a component or an application logic process provides. Pages and 
components can provide values using embedded forms where the user can enter informa- 
tion and submit it. Since a page can contain multiple independent forms, a page can have 
multiple output interfaces as opposed to a single input interface. Similarly, application 
logic processes can offer multiple output interfaces based on values created, calculated 
or retrieved at runtime. 

With the interfaces for all components and pages, the conceptual model (called 
XGuide sitemap) is complete. It serves as a basis for the detailed page and component 
specifications. The next section discusses Web contracts, their internal structuring into 
concerns, and how they can be composed. 



3 Web Contracts and Contract Composition 

A Web contract is a design-level specification of one or multiple Web pages or compo- 
nents that is structured into separate concerns. XGuide currently supports concerns for 
the content (i.e,, information that is offered to the user), the graphical appearance (i.e., 
the layout - the formatting information with which the content is formatted for presen- 
tation), and the application logic (i.e., the functionality that is necessary for providing 
the dynamic interaction with users) of a page. Each concern has its own (XML-based) 
specification vocabulary; the contract itself acts as a container for the concerns. 



1 <xcontract version="l . 0"> 

2 cconcern docElement= "webpage" type=" Content "> <!-- XML schema --> </concern> 

3 cconcern type="AppLogic"> <!-- XML interface definition --> </concern> 

4 

5 <compositionreferences> 

6 creference version=" 1 . 0 " to= "header . contract "> 

7 ccomposition type= " Content "> 

8 coperator elementName= "webpage" position= "beginning" /> 

9 </composition> 

10 ccomposition type="AppLogic"> 

11 cin> 

12 cparam-ref name="page_id"> 

13 coperator type="omit" value= " /webpage/@id" /> 

14 c/param-ref> 

15 c/in> 

16 c/composition> 

17 c/reference> 

18 c/compositionreferences> 

19 c/xcontract> 



Fig. 2. The contract for the programme page of the VIF Web site. 



The content concern specifies the structure and type of the page’s content using an 
XML schema. This concern is used by the content manager to prepare the content for 
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the page and by the layout designer to apply style information to the content elements. 
The application logic concern, on the other hand, captures the input and output interface 
requirements of the page and is used by the programmer. The contract shown in Figure 2 
defines the two concerns in lines 2 and 3 and would include an XML schema and an 
XML-based interface definition in the respective <concern> elements. 

Contracts also form the foundation of XGuide’s concurrent implementation phase. 
They capture all information necessary to implement the various aspects of the page 
independently of each other, i.e., the content manager, layout designer and programmer 
can work in parallel based on the information provided in the contract. 

In the example in Figure 1, all pages reference the header component and thus 
their contracts need to be composed with the header component’s contract. In Figure 2 
the composition information is represented by the <compositionref erences> 
element (lines 5-18). Contract composition works on a per concern basis. Consequently, 
each concern defines its own composition operator. 

3.1 Content Composition 

Since the content concern uses an XML schema to define the structure and the data types 
used for content modeling, composing content concerns can rely on schema composition. 
To keep the operation simple, we currently restrict content composition to extending 
an existing element definition either by inserting a new element at the beginning or 
appending it at the end of an existing element’s content model. 

Figure 2 shows the programme page’s contract that references the header compo- 
nent’s contract. The content composition operator (lines 7-9) defines that the webpage 
element should be extended at the beginning with the referenced header. 

3.2 Application Logic Composition 

The composition of application logic concerns requires a more sophisticated composition 
operation. We distinguish between composition of input interfaces and composition of 
output interfaces. Since output interfaces represent Web forms that are embedded in a 
page, composition of forms is cumulative. This means that if we reference a component 
that contains a form, i.e., has an output interface, we add a new output interface to the 
referencing component. 

For input interfaces, the situation is more complex. Since Web components do not 
exist on the client-side (because they are represented as HTML), the task is to create an 
input interface for the whole page (for page creation at the server side) that subsumes the 
input requirements of all components on the page. As a consequence, the composition 
operation does not deal with the whole input interface but defines a composition operator 
for every parameter in the input interface of a referenced component. We distinguish 
three cases when composing a component input value c, of a component C with the 
input interface of a page P. If c, is not related to any value in P’s input interface, 
it is added to the new input interface ( composition-by-addition ). If the input value Cj 
happens to be already represented in P’s input interface (because the page also needs 
this value as input), the composition operation does not need to modify the input interface 
but merely adds a mapping from the existing value to c, ( composition-by-unification ). 
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Finally, Ci can be directly provided by the content of the embedding page P. In this 
case, Ci is mapped to an XPath expression identifying the value for c, in the content 
of P ( composition-by-omission ). Figure 2 gives an example for this case (see lines 10- 
16). The value for the page Jd input parameter of the header component is provided by 
evaluating the /webpcige/@ id XPath expression on the embedding page. Thus though the 
header component requires a page Ad input parameter, it does not appear in the page’s 
input interface. For more information on the XGuide contract calculus see [3]. 

In addition to the currently supported content and application logic concerns, we are 
working on two additional concern plug-ins capturing the specification information for 
meta-data and access control to page content based on user identity. Each concern plug- 
in again consists of an XML language to express the specification information and the 
corresponding composition operators. Given the contracts for all pages, developers can 
start implementing the specified concerns. Because of the strict separation of concerns in 
the contracts, they can even do so in parallel and assemble the concern implementations 
at the end. 

To support the development process, we provide a prototype tool called XSuite that 
supports the XGuide method. XSuite is an Eclipse plug-in and supports visual modeling 
via Microsoft Visio. It provides dialogs and wizards to create contracts for all artifacts in 
the XGuide sitemap, to define pages based on the created contracts, to implement these 
pages in the My XML [4] implementation technology (other implementation technolo- 
gies are supported via a plug-in mechanism), and to test and deploy the implemented 
pages. 

4 Conclusion 

This paper describes how we apply the concept of design-by-contract to the field of Web 
engineering and proposes the use of specifications (Web contracts). It demonstrates the 
use of contracts in the XGuide approach and shows how they influence the composition 
of Web components and facilitate a parallel implementation. A simple example from 
deploying XGuide in the VIF 2003 case study is shown. The strict separation of concerns 
adds additional benefits through an increased reuse potential and eased maintenance. 
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Abstract. Web engineering can be seen as the application of software 
engineering in a particular environment - the Web. Its most important 
aspect is the orientation towards distributed systems and their current 
technology. Current specification techniques like UML do not cover con- 
currency, distribution and deployment in a sufficient way. 

We propose a modeling technique based on high-level Petri nets, called 
reference nets. They naturally allow for the modeling of processes. Addi- 
tionally we introduce a framework that covers the other aspects of Web 
application modeling. Finally we introduce a tool set based on Renew, 
a tool for reference nets, to support these modeling approach. 

Keywords: High-level Petri nets, nets within nets, reference nets, Re- 
new, Web service, modeling, specification, Mulan 



1 Introduction 

Web Engineering as it is seen here is the attempt to perform high-quality soft- 
ware engineering in distributed systems for autonomous (often independent) 
participants. These systems are characterized by distribution, concurrency and 
often autonomy; the term Web indicates the current embedding of this process 
in the Internet as the underlying execution platform. 

One important area in Web engineering is the area of Web services which 
are easier to handle than agents in general but which still contain the aspects 
of distribution and adaptability. Web services are self-describing, self-contained 
encapsulated applications. The modeling approaches currently often applied in 
practice are structured analysis and object-oriented analysis. The first one is 
generally not considered to be sufficient for future applications, while there is a 
large amount of research effort going into the development of the second and its 
main representatives the set of modeling techniques: UML [7] (Unified Modeling 
Language) . 

Especially the open problems of Web engineering, however, can not be solved 
by using a relatively simple concept like e.g. State charts for concurrency, auton- 
omy and distribution. When building distributed and concurrent applications a 
true concurrency semantics has to be used to cover the problems which can be 
encountered at any time and level of the implementation. Here, an interleaving 
semantics is no longer sufficient. 
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Distributed applications might no longer behave like an object, but might 
show an autonomous behavior. This leads to a modeling approach that is closely 
related to the agent concepts. Another aspect is the execution of distributed 
tasks. Here the support of workflow-like concepts is crucial. The UML does not 
offer a clear intuitive modeling technique for these concepts and so a lot of the 
specification is often implicitly done by the code. 

One of the techniques that should be taken into closer consideration for mod- 
eling Web service applications are higher-level Petri nets naturally allowing to 
model distribution and concurrency. With Mulan [5] a proposal for the inte- 
gration of agents and Petri nets has been made. 

By adapting Mulan for the Web service context, we present a modeling 
technique that is designed to solve the central problems of Web engineering on a 
conceptual level. The MuLAN-framework allows to build specifications that cover 
the above mentioned aspects of distributed applications. Our proposal extends 
the usual set of techniques which is available for Web engineers and provides a 
powerful architectural basis and structuring mechanism for web applications. 

In the following section the underlying concepts and tools used in our ap- 
proach are introduced. Section 3 explains the use of our approach for Web service 
orchestration whereas section 4 shows its application to Web service system de- 
sign and adapts the existing Mulan agent framework for Web services. Section 
5 summarizes the paper and gives a short outlook. 



2 Reference Nets 

The modeling approach described in the sequel is strongly supported by a tool 
called Renew [9]. Renew is a Petri net simulator implemented in Java which 
allows for the construction and the execution of reference nets. 

Reference nets enrich the Coloured Petri net (CPN) formalism [4] by adding 
concept of nets-witlrin-nets introduced by [10]. Nets-within-nets represent a con- 
cept of having tokens within a Petri net as net instances. This way nets can have 
multiple object nets within a single system net as tokens. The communication 
between these nets is done through synchronous channels. They allow for the ex- 
change of parameters between the net instances and within a single net instance. 
This way they offer a way to communicate between net instances. Those object 
nets can be dynamically instantiated from a template. Multiple net instances can 
be generated at runtime having different parameters. By having transitions in- 
scribed with arbitrary Java code, Renew additionally offers an easy integration 
of Java and can therefore as well execute Web services. 

3 Composition of Web Services 

A common approach to build new software applications is to identify the needed 
functionalities that must be provided by modular units of the application and 
than implement and/or reuse components matching certain requirements. In the 
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context of distributed systems a growing trend is to build such software compo- 
nents platform-independent and make them available in a distributed environ- 
ment. On this basis new Web applications can be largely composed/ assembled 
from a set of appropriate distributed components, called Web services. 

A natural approach to model the order of the interactions needed to fulfill a 
complex business collaboration is the use of some kind of business process model. 
This can be described by a variety of different process markup languages. They 
specify the control and data flow of a business process or the interaction protocols 
describing the message exchange between the parties involved. A proposal for a 
model to specify dynamic composition of Web services based on Java, reference 
nets and current web languages is presented in [8] , where the tasks for the real- 
ization of a new business service, like interactions or message transformations, 
are modeled by inscriptions of transitions in a reference net and a platform for 
the model based composition of Web services is implemented by the use of the 
Petri net simulator Renew. 

Petri nets offer a single formalism and a visual modeling technique for control 
flows as well as for the data flow. As a modeling technique for workflows, Petri 
nets have been thoroughly investigated [1]. Apart from their graphical visual- 
ization, they offer for some net variants means to verify properties of workflows 
like liveness or the absence of deadlocks [2] . 

Within the component oriented view we provide a conceptual framework and 
a tool set to dynamically compose Web services. Allowing for the execution of 
Petri nets, Renew offers a tight integration of the specification of the control 
flow and its implementation. This is further supported by the formal semantics of 
Petri nets. As shown in [1] and demonstrated by the implementation in [3] Petri 
nets are powerful enough to represent all workflow concepts, which is equally 
true for the control flow concepts used in Web service description. 

4 Requirements for Web Service Architectures 

Besides the control flow of Web services we consider the physical or logical 
location of Web services to be another important area of modeling. Web services 
are hosted on some kind of Web server which is again located on a platform. 

The access to Web services might fail or Web services might not be able to 
handle certain requests. In these cases the caller has to be able to dynamically 
switch to another service offering similar functionality. This is closely related to 
agents with respect to their autonomy and adaptation, what justifies a model- 
ing approach for Web services that was adopted from the agent research area. 
Mulan [5] is a Petri net based modeling approach for multi-agent systems. Fig- 
ure 1 depicts the basic architecture adopted for Web services. Here we see that 
Web services are located on some kind of physical host that they are deployed 
upon and that they have a logical platform (which might range over multiple 
hosts). The protocols specifying their behavior are again modeled as Petri nets. 
The ZOOM lines show a refinement from one layer to the succeeding layer. Web 
services might fail or might no longer be available. In these cases the caller can 
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Web service network 







Fig. 1 . Web service architecture 



ask directory services on the (logical) platform for another similar service. The 
four layers distinguish different aspects of Web service applications. 

The first layer consists of the hosts where one or more Web service containers 
might be deployed. These hosts offer the basic communication infrastructure. 

On top of this layer we have the Web service container offering more specific 
communication based on SOAP and WSDL as well as access to internal services 
on the one hand and means to deploy and remove Web services on the other 
hand. A Web service implementation makes use of this layer to communicate 
to the outside world (e.g. to other Web services) and to internal services (e.g. 
databases and repositories). 

The actual Web service implementation is located on top of this layer. Calls 
on the container are forwarded to the appropriate Web service. The Web service 
might then call other Web services or access internal services to handle the 
request. 

The control flow to handle the request represents the fourth layer. An incom- 
ing request triggers a certain control flow and then passes back the results. 

The architecture in figure 1 reflects the different aspects of Web services. 
Each of these layers can be further refined if necessary. This allows for arbitrary 
granularity in the modeling process. The internal and external behavior of the 
Web services is modeled by protocol nets. These can also be generated out of 
OWL-S and BPEL4WS descriptions [6]. 

5 Summary and Outlook 

UML can be seen as the lingua franca. Its different modeling techniques support 
different views on a system. During the last years considerable effort has been put 
into the development of the techniques, tools and methods that are necessary for 
software development. A formal semantics, however, has often been neglected, 
and key features of current Web applications are not supported: concurrency, 
autonomy and adaptivity. 

Even Petri nets equipped with a precise semantics do not provide sufficient 
means to cover these features in their models. What is missing is a software 
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architecture. Here the MuLAN-framework provides a conceptual and practical 
solution. The very high-level Petri nets are combined with agent concepts to 
integrate the advantages of both areas. 

In this paper, a formal modeling technique has been adopted to fit the needs 
when modeling complex web applications. In practical work the suitability of the 
facets of the approach have been demonstrated. The tool set covers important 
aspects of web engineering. Due to the complex domain, the facets have been 
developed in isolation, demonstrating their respective strength. During student 
projects with 20 to 50 participants the approach has been applied successfully 
for the last three years in the area of agent-oriented software development. 

We will continue to extend Renew as the basic tool and we will focus on 
the Petri net aspects in combination with agents and their Web technological 
embedding. On the other hand we take care to use web and software engineer- 
ing experience wherever it is possible. As demonstrated in [8] non-agent based 
structuring is investigated. In and [6] the same is shown for the Semantic web. 



References 

1. Wil M.P. van der Aalst and Arthur H.M. ter Hofstede. Workflow pattern homepage. 
Technical report, URL: http://tmitwww.tm.tue.nl/research/patterns/, 2003. 

2. C. Girault and R. Valk. Petri Nets for Systems Engineering - A Guide to Modeling, 
Verification, and Applications. Springer- Verlag, 2003. 

3. Thomas Jacob. Implementierung einer sicheren und rollenbasierten Workflow- 
Management-Komponente fur ein Petrinetzwerkzeug. Diplomarbeit, Universitat 
Hamburg, Fachbereich Informatik, 2002. 

4. K. Jensen. High Level Petri Nets. In A. Pagoni and G. Rozenberg, editors, Ap- 
plications and Theory of Petri Nets, number 66 in Informatik Fachberichte, pages 
166-180, Berlin, 1983. Springer- Verlag. 

5. Michael Kohler, Daniel Moldt, and Heiko Rolkc. Modelling the structure and 
behaviour of Petri net agents. In Proc. of 22nd International Conf. on Applications 
and Theory of Petri Nets 2001 (ICATPN 2001) / J.-M. Colom, M. Koutny (Eds.), 
Newcastle upon Tyne, UK, pages 224-242. Lecture Notes in Computer Science 2075, 
edited by G. Goos, J. Hartmanis and J. van Leuwen, Springer, June 2001. 

6. Daniel Moldt and Jan Ortmann. DaGen: A Tool for Automatic Translation from 
DAML-S to High-level Petri Nets, 2004. to appear, excepted as tool paper to FASE 
2004. 

7. Object Management Group. UML resource page. Technical report, 2004. 

8. Sven Offermann. Ein Referenz-Netz-basiertes Modcll zur dynamischen Komposi- 
tion von Web Services. Master’s thesis, Universitat Hamburg, Fachbereich Infor- 
matik, 2003. 

9. Renew - the reference net workshop. URL http : //www . renew. de/. Reference to 
the program, the source code and the documentation of the Renew simulator. 

10. R. Valk. Petri nets as token objects - An introduction to elementary object nets. In 
J. Desel and M. Silva, editors, Proc. Application and Theory of Petri Nets, Lisbon, 
Portugal, number 1420 in LNCS, pages 1-25, Berlin, 1998. Springer- Verlag. 




Extending Navigation Modelling to Support Content 
Aggregation in Web Sites* 



Pedro Valderas, Joan Fons, and Vicente Pelechano 



Department of Information Systems and Computation 
Polytechnic University of Valencia 
Cam! de Vera s/n 
46022 Valencia, Spain 

{pvalderas, jjfons, pele}@dsic .upv. es 



Abstract. Currently, web sites are integrators of heterogeneous information and 
services that are oriented to providing content aggregation. From a Model- 
Driven perspective, web applications need conceptual mechanisms that make it 
easy to describe, manage and reuse contents and services in order to deal with 
content aggregation at a higher level of abstraction. Our work presents 
conceptual modelling techniques that extend the OOWS navigational modelling 
by refining the navigational context definition and introducing the concept of 
information abstraction unit to specify the contents of web applications of this 
kind. These new abstractions provide powerful reuse mechanisms that produce 
considerable benefits because both development time and effort can be reduced. 
Finally, the paper presents some ideas to implement content aggregation taking 
these enhanced navigational models as input. 



1 Introduction 

Web sites are assuming a greater leading role in Internet. Web sites like Yahoo 
(www.yahoo.com), MSN (www.msn.com), Lycos (www.lycos.com) or Terra 
(www.terra.es) are ranked among the web applications that receive the most visits. 
The main characteristic of applications of this type consists of providing the users 
with web pages that are built from aggregation of contents. These pages provide a lot 
of information and services on issues of a diverse nature. Figure 1 shows the home 
page of the Terra web site that is build from the aggregation of diverse contents or 
information blocks. 

From a methodological point of view, the most outstanding approaches (OOHDM 
[2], WebML [5], WSDM [1], etc.) focus their efforts on defining web applications 
from conceptual models that allow them to specify hypermedial and functional 
requirements. The mechanisms for hypermedia modelling allow the analyst (1) to 
define web pages as conceptual schema views and (2) to interconnect these views to 
define the navigational structure of the web application. Flowever, considering a web 
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page only as a view of the conceptual schema makes difficult to specify aggregation 
of contents, where web pages are built as a composite of several conceptual schema 
views (see each numbered area in Figure 1). In this way, although some of the 
approaches mentioned above provide support in the design and/or implementation 
steps, none of them explicitly supports the specification of web applications with 
aggregation of contents in the conceptual modelling step. 

In this work, we present 
conceptual modelling techniques 
that extend the OOWS (Object- 
Oriented Web Solution) [4] 
navigational modelling by 
refining the navigational context 
definition and introducing the 
concept of information 
abstraction unit in order to 
specify the contents for web 
applications of this kind. This 
new expressive capacity makes it 
easy to describe at a higher level 
of abstraction of web 
applications with content 
aggregation. It also provides Fig. 1. Terra Home Page 

powerful reuse mechanisms that 

can reduce development time and effort. Following a Model-Driven Development 
(MDD [6]) approach, the paper presents some ideas to implement web applications 
that support content aggregation by taking these enhanced navigational models as 
input. 

This paper is organized in the following way: section 2 introduces the OOWS 
method which presents the proposed extensions at a conceptual modelling level using 
the Terra web site as a case study. Conclusions and future works are commented on in 
section 3. 




2 Content Aggregation in OOWS 

OOWS (Object-Oriented Web Solutions) [3] is the extension of an object-oriented 
software production method (OO-Method [4]) that introduces the required 
expressivity to capture the navigational requirements of web applications. In order to 
do this, OOWS introduces the Navigational Model that allows for capturing the 
navigation semantics in two steps: the “Authoring-in-the-large” (global view) and the 
“Authoring-in-the-small” (detailed view). 

The Authoring-in-the-large step refers to the specification and design of global and 
structural aspects of the web application. These requirements are specified in a 
Navigational Map that provides a specific kind of user with its system view. It is 
represented using a directed graph whose nodes denote navigational contexts and 
whose arcs denote navigational links or valid navigational paths. 

These navigational contexts (graphically represented as UML packages stereotyped 
with the «context» keyword) represent the user interaction units that provide a set of 
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cohesive data and operations. The navigational links (navigational map arcs) represent 
context accessibility or “navigational paths”. Figure 2 shows a piece of the 
navigational map for the Terra web site related to an anonymous Internet user. 
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Fig. 2. Terra navigational map 




Fig. 3. Home navigational context and News and Products lAUs 



The “Authoring-in-the-small” step refers to the detailed specification of the 
contents of the navigational contexts. The following section presents how these 
contexts must be defined in order to achieve content aggregation. 



2.1 Navigational Contexts with Content Aggregation 

In order to support content aggregation, navigational contexts should be considered as 
user interaction units which provide access to several information abstraction units. 
An information abstraction unit (IAU), stereotyped with the «iau» keyword, 
represents a specific view on the class diagram. Each IAU is made up of a set of 
navigational classes that represent class views (including attributes and operations). 
These classes are stereotyped with the «view» keyword. Moreover, selection filters 
(expressions that allow us to select specific objects) can be defined in OCL upon an 
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object population which has been retrieved by a navigational class (see Figure 3, 
“self.latestRelease=True” at Products class and “self.Date=today()” at News class). 

Navigational classes must be related by unidirectional binary relationships, called 
navigational relationships. They are defined over existing aggregation/association/ 
composition or specialization/generalization relationships. 

Figure 3 shows an example of the Home navigational context. This context is 
defined using five IAUs: Top News, News, Products, Leisure and Advertising. Figure 
3 also shows the News IAU definition (that provides the news of the day) and the 
Products IAU definition (that provides information about the latest product releases). 
In Figure 1 we can see the implementation of this navigational context. In areas 1 , 2, 
3, 4 and 5 we can see the IAUs implementation corresponding each one to a different 
section of the home page. 

2.2 Reuse Mechanisms 

Introducing IAUs into the OOWS navigational model allows us to provide some 
mechanisms to reuse contents at a high level of abstraction. These mechanisms allow 
us (1) to specify new navigational contexts and/or (2) to specify new IAUs taking a 
predefined IAU as an input. 

1. Specifying New Navigational Contexts : an IAU 
specified in a navigational context can be used to define 
other new contexts by means of a reference to it in the 
new definition. In this way, a new navigational context 
can be made up of: (1) new IAUs (mandatory option for 
the navigational context which is specified first) or (2) 

IAUs specified in other navigational contexts. 

2. Specifying New IAUs through specialization : new 
IAUs can be defined taking an already defined one as a 
basis. We do this by extending the IAU definition by 

means of specialization mechanisms: the specialized 
IAU inherits a parent IAU definition. It can be refined 

to adapt the retrieved information, 
services and navigation capabilities 
to the needs of a particular 
navigational context, carrying out 
the following operations: Adding or 
removing attributes, adding or 
removing services, adding or 
removing navigational classes, 
adding or removing navigational 
relationships and adding, removing 
or redefining population selection 
filters. 

IAU specialization is performed 
by means of some kind of is_a operator, allowing the analyst to refine the specialized 
IAU using the previous operations. Figure 4 shows the Top News IAU definition. This 
IAU has been specialized from the News IAU (see the top of the Figure 4). In this 





Fig. 5. Top News and News IAUs 
implementation. 




Fig. 4. Top News IAU 
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case, it has been refined by adding the Image attribute and redefining its population 
selection filter (only the last news item is retrieved at this moment). These 
refinements have been highlighted in Figure 4. Figure 5 shows sections 1 and 2 of 
Figure 1 corresponding to the implementation for both Top News and News IAUs. 



3 Conclusions 

In this paper we have proposed a solution from the conceptual modelling perspective 
in order to give methodological support for the development of web applications with 
content aggregation. 

This solution redefines the navigational context as a conceptual abstraction that can 
be built using multiple information abstraction units (IAUs). Each IAU represents at 
the conceptual modelling step a specific content and the use of IAUs allows us to 
carry out content reuse mechanisms. 

As future work, it would be necessary both to extend the presentation patterns that 
OOWS Presentation Model provides to attach them to the IAUs and to detect 
presentation relationships and dependencies between different IAUs in the same 
navigational context. 
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Abstract. As the Web becomes a platform for implementing B2B applications, 
the need arises of extending Web conceptual modeling from data-centric appli- 
cations to data- and process-centric applications. New primitives must be put in 
place to implement workflows describing business processes. In this context, 
new problems about process safety arise, due to the loose control on Web cli- 
ents. Indeed, user behavior can generate dangerous incoherencies for the exe- 
cution of processes. This paper presents a proposal of workflow-enabling 
primitives for Web applications, and a high level approach to the management 
of exceptions that occurs during execution of processes. We present a classifi- 
cation of exceptions that can occur inside workflow-based Web applications, 
and recovery policies to retrieve coherent status and data after an exception. An 
implementation experience is briefly presented too. 



1 Introduction 

In recent years, the Web is more and more being used as the implementation platform 
for B2B applications, whose goal is not only the navigation of content, but also sup- 
porting business processes, content management, value-added services and so on. 
Conceptual modeling expertise from other fields (database, object-orientated pro- 
gramming, hypermedia applications) has been widely recognized as valid starting 
point for defining conceptual aids for Web application development too [7]. The first 
generation of conceptual models for the Web [1. 2, 5, 6] essentially focus on captur- 
ing the structure of data to be published, and the navigation primitives, represented by 
such concepts as pages, content nodes, and links. 

To cover business processes support, a second generation of conceptual models is 
required. These new models should cope with process and workflow modeling, sup- 
port Web service interaction, and integrate data-centric and process-centric modeling 
primitives into a mix suited to the development of advanced B2B Web applications. 
In this context, it is important to address the critical cases that can occur in the enact- 
ment of business processes on a Web-based platform. 

This paper presents an extension to a first-generation Web modeling language [5, 6] 
to support the specification, design and implementation of B2B applications, and 
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offers an high-level analysis of critic aspects and exception management issues within 
Web applications exploiting business processes. Exceptions that can happen in a Web 
based application have peculiar characteristics with respect to traditional workflow 
applications. This is due to two main aspects: (i) interaction options provided by 
browser-based interfaces are very powerful, but they are more oriented to free navi- 
gation than strict processes adherence(e.g., user is enabled to jump back and forth on 
navigated pages, thus introducing dangerous repetition of process activities); (ii) user 
cannot be forced to perform any action or task (e.g., he can stand on a page for long 
time, or even close the browser and disconnect at any time). 

Our approach is lightweight: we are interested in extending Web modeling to cope 
with process and exception modeling, not to adapt workflow management systems to 
the Web; about exceptions, we aim at defining a modeling paradigm for critical cases, 
not to build transactional systems or low level exception handling mechanisms. 

Many works have addressed the problem of exception interception and compensation. 
They mainly studied transactional properties for activities, which is not in our scope. 
However, some works deals with weaker properties. For example, [8] is based on the 
concept of spheres, to make use of only those transactional properties that are actually 
needed; [12] is one of the first works that address the problem in the Web context. 

The paper is organized as follows: Section 2 briefly outlines the main concepts about 
workflow and Web application modeling; Section 3 introduces the study of exception 
and critical situations that can occur in the execution of processes on the Web; Sec- 
tion 4 presents our approach to management and recovery of exceptional situations 
within process execution; finally Section 5 reviews implementation experience and 
Section 6 draws some conclusions and presents our ongoing and future work. 



2 Conceptual Modeling of Web Applications and Workflows 

Conceptual design consists in high-level, platform-independent specification of the 
application, which can be used to drive the subsequent implementation phase. In this 
section we focus on two aspects of conceptual design: (i) Web application design, 
briefly describing the WebML model, that will be used in the sequel to describe our 
proposals; (ii) Workflow modeling concepts and primitives. 

It is important to point out that, although the paper uses the WebML notation to de- 
scribe our contribution, the proposed approach is independent from the specific lan- 
guage or notation that is adopted. Our approach to conceptual design relies on the 
following guidelines: an Entity-Relationship diagram models the data stored, ma- 
nipulated, and exchanged by the application actors, plus the metadata required for the 
management of the business processes; process diagrams are treated as a higher-level 
specification and are used to derive a set of hypertext models that "realizes" them. 
These hypertext models belong to the site views of the user groups involved in the 
process and must offer them the interface needed for performing their activities. 
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2.1 Process Modeling 

For specifying processes, we adopt the terminology and notation defined by the 
Workflow Management Coalition [16], which provides a workflow model based on 
the concepts of Process (the description of the supported workflow), Case (a process 
instance), Activity (the elementary unit of work composing a process). Activity in- 
stance (an instantiation of an activity within a case), Actor (a user role intervening in 
the process), and Constraint (logical precedence among activities and rules enabling 
activities execution). Processes can be internally structured using a variety of con- 
structs: sequences of activities, AND-splits (a single thread of control splits into two 
or more independent threads), AND-joins (blocking convergence point of two or 
more parallel activities), OR-splits (point in which one among multiple alternative 
branches is taken), OR-joins (non-blocking convergence point), iterations for repeat- 
ing the execution of one or more activities, pre- and post-conditions (entry and exit 
criteria to and from a particular activity). 

Fig. 1 exemplifies a WfMC workflow specifying the process of online purchase, 
payment and delivery of goods. The customer can choose the products to purchase, 
then submits his payment information. At this point, two parallel tasks are executed 
by the seller employees: the warehouse manager registers the shipping of the order, 
and a secretary prepares a bill to be sent to the customer. 




Performed by: 
Secretary 



Fig. 1. Workflow diagram of the refunding request process 



2.2 Hypertext Modeling 

For hypertext modeling, we use the WebML notation[5, 6, 14], a conceptual language 
for specifying Web applications developed on top of database content described by a 
E-R diagram. A WebML schema consists of one or more site views, expressing the 
Web interfaces that allow the different user roles to browse or manipulate the data 
specified in the underlying E-R schema. A site view contains pages, possibly clus- 
tered in areas, typically representing independent sections of the site. Pages enclose 
content units, representing atomic pieces of information to be published (e.g., indexes 
listing items from which the user may select a particular object, details of a single 
object, entry forms, and so on); content units may have a selector, which is a predi- 
cate identifying the entity instances to be extracted from the underlying database and 
displayed by the unit. Pages and units can be connected through links of different 
types to express all possible navigation. 
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Besides content publishing, WebML allows specifying operations, like the filling of a 
shopping cart or the update of content. Basic data update operations are: the creation, 
modification and deletion of instances of an entity, or the creation and deletion of 
instances of a relationship. Operations do not display data and are placed outside of 
pages; user-defined operations can be specified (e.g., e-mail sending, e-payment, ...), 
and operation chains are allowed too. 

Fig. 2 shows a simplified version of the two areas of the Customer site view of the e- 
commerce site example, whose workflow have been illustrated in Fig. 1 : the Products 
area allows guests to browse products, by selecting in the Home page the product 
group from an index ( ProductGroups ). Once a group is selected, all the products of 
that group are shown in page Products. The Mailing List Subscription area allows the 
user to subscribe to a mailing list through a form. The submitted data are used to 
modify the profile of the User, by means of a modify operation called Modify Subscr, 
which updates the instance of entity User currently logged. 




Fig. 2. WebML site view diagram featuring areas, pages, content units, and operations. 



2.3 Extending Hypertext Modeling to Capture Processes 

In the specification of a Web application supporting business processes[3], the data 
model, which is normally used to describe the domain objects, is extended with user- 
related and workflow-related data, and the hypertext model is enriched by a set of 
primitives enabling workflow dependent content of pages and navigation. 

Process metadata. Data modeling is extended with the metadata used to represent the 
runtime evolution of processes as shown in Fig. 3. The schema includes entities rep- 
resenting the elements of a WfMC process model, and relationships expressing the 
semantic connections between the process elements. 

Entity Process is associated with entity ActivityType, to represent the classes of ac- 
tivities that can be executed in a process. Entity Case denotes an instance of a proc- 
ess, whose status can be: initiated, active, or completed. Entity Activitylnstance de- 
notes the occurrence of an activity, whose current status can be: inactive, active and 
completed. Entities User and Group represent the workflow actors, as individual 
users organized within groups (or roles). A user may belong to different groups, and 
one of these groups is set as his default group, to facilitate access control when the 
user logs in. Activities are "assigned to" user groups: this means that users of that 
group can perform the activity. Instead, concrete activity instances are "assigned to" 



Exception Handling Within Workflow-Based Web Applications 



107 



individual users, who actually perform them. If needed, the model can be enriched at 
will with new relationships to represent more complex assignment rules. 




Fig. 3. Data model incorporating workflow concepts and exception handling information 



Application data is described by a usual E-R model representing information involved 
in the current application. In our example, as depicted in the boxed part of Fig. 3, we 
model a catalog (in which each Product belongs to a ProductGroup), the Orders that 
the user submits and the Payment details. Orders are assigned to Activity Instances in 
which will be processed, whilst Payments are connected to the Activity Instances in 
which they are created. These relationships associate metadata concepts to application 
information. In general, the designer can specify an arbitrary number of relationships 
between the application data and the workflow data, which may be required to con- 
nect the activities to the data items they use. Note that minimum cardinality of these 
relationships is typically 0, since in most cases each activity instance is not associated 
to all the application data, but only to a very small set of objects. 

This schema already includes metadata for supporting exception handling informa- 
tion. Such data are represented in bold face. In particular, the Created relationship 
and the CurrentStep entity are needed for supporting recovery policies for exceptions. 
Their use will be explained in the sequel of the paper. 

Workflow hypertext primitives. In order to enact the process, some workflow- 
specialized hypertext primitives are also necessary to design interfaces capable of 
producing and consuming such metadata. At this purpose, a few additional primitives 
are introduced in WebML for updating process data as a result of activity execution, 
for accessing the data associated with a specific activity instance, and for expressing 
the assignment of data objects to an activity instance eventually to be executed. 

The portion of hypertext devoted to the execution of an activity must be enclosed 
between the two workflow-related operations shown in Fig. 4(a): start activity and 
end activity. These operations are triggered respectively by incoming and outgoing 
links of the activity and have the side effect of updating the workflow data. Specifi- 
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cally, starting an activity implies creating an activity instance, recording the activity 
instance activation timestamp, connecting the activity instance to the current case 
(relationship PartOf), to the current user (relationship AssignedTo), and to the proper 
activity type, and setting the status of the activity instance to "active". Symmetrically, 
ending an activity implies setting the status to "completed" and recording the time- 
stamp. 



'Start Activity 




( End Activity^ 


O 

V J 




• 



ActivityName ActivityName 

(a) 




[Case=CurrentCase] 

(b) (c) (d) 



Fig. 4. Start Activity and End Activity operations (a); workflow-aware content unit notation(b); 
graphical notation of the Assign operation (c) and of the conditional operation (d) 



The Start Activity operation can also be marked as the starting case activity, when the 
activity to start is the first one of the entire process; dually, the End Activity operation 
can be tagged as the end of the case, thus recording the general status of the process. 
Workflow-aware content units can be used for retrieving the data objects related to a 
particular activity. These units are like the regular WebML content unit but are tagged 
with a "W" symbol denoting a simplified syntax for their selector, which shortens the 
expression of predicates involving both application data and workflow data. For ex- 
ample, Fig. 4 (b) shows a workflow-aware index unit that retrieves all the instances of 
entity Payment that have been assigned to an activity of type “Billing”. 

The assign operation is a WebML operation unit that connects application object(s) 
to an activity instance, for which an activity type, a case and possibly a user are speci- 
fied. Fig. 4(c) shows the graphical representation of the assign operation, which as- 
signs a Payment to the activity called "Billing" for the current process case. 

The navigation of a hypertext may need to be conditioned by the status of activities, 
to reflect the constraints imposed by the workflow. Two dedicated operations called if 
(see Fig. 4(d)) and switch operations allow conditional navigation, performing the 
necessary status tests and deciding the destination of a navigable link. 

Mapping rules have been defined from WfMC-based workflow description to 
WebML hypertexts enhanced with workflow primitives [3]. 



2.4 Fine Grained Description of Activities 

To study in a simple and effective way the exception handling problem, we define 
some new concepts that describe the structure of activities. 

We call step a hypertext page belonging to an activity. Steps are univocally num- 
bered within an activity. Between two subsequent steps there can be a chain of opera- 
tions, which is not relevant for our purposes. Indeed, since we do not consider server- 
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side failures, a chain of operations can be seen as an atomic element that never fails 
(server-side failure is addressed by standard WebML mechanisms, like KO links [6]). 
We define the current step of an active activity as the last page that the server has 
generated after a request by the client. This information is stored into the CurrentStep 
entity of the workflow metadata schema (Fig. 3). Within a process case, it is always 
possible to retrieve the currently active activities, and for each of them the current 
step. 

The current step has 2 important properties: (i) it is always uniquely defined for an 
active activity; (ii) it gives us a correct idea of the progress of the activity. 

It is important to notice that if the client uses the back and forward buttons of the 
browser, the current step of the activity does not change, since the client does not 
make any request to the server. Moreover, by clicking the back button we do not roll 
back the operations between consecutive steps, we just reload an old page. 



3 Critical Situations and Exception 

Within the execution of a process, exceptional situations can occur, due either to user 
behavior or to system failures. We define a critical situation as an incorrect browsing 
behavior of the user ( user-generated exceptions) or a technical failure of the system 
(system failures). 



3.1 User-Generated Exceptions 

This section presents the critical situations that can arise from wrong browsing be- 
havior. For Web context, this problem is much more relevant than for traditional 
applications. The most evident examples are back and forward buttons of a Web 
browser, that allow the user to explore the hypertext of the Web application in a free 
way, while a workflow scenario has usually a strictly forced execution/ navigation 
structure, and its steps must be executed in the proper order. Moreover, the user is 
able to jump without restrictions from an application to another. Back and forward 
buttons let the user go outside the pages of an activity still active or move back to a 
completed activity and try to resume its execution. With respect to workflow activi- 
ties, improper browsing can be of three types: 

(i) improper inbound browsing: the user gets into a workflow activity without exe- 
cuting the Start activity operation, for example by clicking repeatedly on the back 
browser button, until a previously executed activity is reached; 

(ii) improper outbound browsing: the user, during the execution of an activity, fol- 
lows a wrong navigational path, exiting the activity without passing by the End activ- 
ity operation. In this case the user leaves the pages of the current activity, either by 
pressing repeatedly the back browser button or by following a landmark link (i.e., a 
link which is always clickable within the whole Web application). In this way, the 
user can potentially start an arbitrary number of activities, since he can try to start a 
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new activity beside the current one. Moreover, the user left an activity in status Ac- 
tive, which cannot proceed, and thus remains halted; 

(iii) improper internal browsing: the user, during the execution of an activity, 
presses the back button of the browser one or more times reaching a previous page of 
the same activity, and then clicks on a link, trying to repeat part of the activity. In this 
way, the user is in a page that is different from the current step of the activity, since 
the page from which the user resumes the browsing is different from the last page 
requested to the server; 

(iv) wait: the user does not request a page to the server for a given amount of time, 
after which a timeout expires and the user session ends up. A Session End exception 
is generated, and this behavior collapses in a system failure. 



3.2 System Failures 

System failures can occur both at client and at server side. Client-side failures are 
problems that are generated by system breakdown, that is either a client crash or a 
network failure. We do not consider server-side failures, since this problem for Web- 
based workflows can be addressed in the same way of traditional workflow systems, 
and several recovery theories and techniques already exist for this context (e.g., rules 
based on active rules [4]). System failures result in a Session end exception at server- 
side. To discover client-side failures, HTTP session is a standard technique employed 
in Web applications. After a session has been established, a network failure or a client 
failure will result either in the client not performing a request to the server for a given 
amount of time, or in the server being unable to send the response back to the client. 
When the server recognizes that the client is no more reachable, it will end up the user 
session: client-side failures can be captured at application level by generating a Ses- 
sion End exception. In this sense, client failure and network failure will be indistin- 
guishable and will be collectively denoted also as Crash situations. 

After a crash situation the activity instance executed by the user remains in Active 
status, but is not completed. This means that the activity execution cannot proceed, 
since the user lost his session, and if he tries to login he can only see the activities that 
are in Inactive status (ready to be executed). Typically he is not allowed to perform 
activities potentially in execution (i.e., in Active status). 

If the activity instance is not recovered, the whole process case will possibly be 
stopped, if there are other activities waiting for the completion of the crashed one. 

A thrown Session End exception will help to track the crash for later recovery 



3.3 Inconsistencies 

Data and process inconsistencies can arise from system failure and incorrect browsing 
behavior. Each of them will be addressed with a different approach: 

1) activity/process halt: one or more activities (and the processes they belong to) get 
halted and cannot be resumed or concluded by the user. These problems are de- 
tected after they take place and are recovered by means of appropriate policies; 
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2) inconsistent database: one or more database tuples are created or destroyed in an 
unexpected way, resulting in an inconsistent database and workflow application; 
these problems are caused by incorrect browsing behavior, and will be handled in 
a preventive way, by detecting the user faults and generating an exception before 
they result in a failure. 



4 Exceptions and Recovery Policies 

As we have seen in previous sections, if a critical situation occurs, the workflow ap- 
plication might be in an inconsistent state due to the presence of a halted activity, i.e. 
an activity in status Active that cannot proceed. The need arises to recover the halted 
activity to bring the workflow application back to a correct state and let the process 
execution proceed. To address the problem, we define the concepts of exception and 
recovery policy. 



4.1 Exceptions 

To manage critical situations and to prevent/recover inconsistencies, we introduce the 
concept of exception. An exception is an event that is thrown by the system, as a 
consequence of a critical situation that is occurred. 

An exception is either synchronous, if it is thrown after a page request, or asynchro- 
nous, if it is not tied to a page request but can occur independently. In case of syn- 
chronous exceptions, the user navigation can be immediately affected since the server 
can decide to provide the user with a different page depending on the caught excep- 
tion. On the other hand, the only asynchronous exception that we will consider is 
Session End. It cannot influence immediately the user browsing, since he already 
disconnected from the application (his session is no more valid). Table 1 resumes the 
characteristics of exception types. 



Table 1. Types and properties of the exceptions 



Exception Type 


Session Status 


Addressed Problem 


Asynchronous 


Inactive 


Technical Failure 
Incorrect Browsing Behavior 


Synchronous 


Active 


Incorrect Browsing Behavior 



Exceptions to be managed in order to guarantee the correctness of workflow-based 
Web applications are the following: 

1) Session End: the user disconnected the client, or a failure happened on the net- 
work or at client-side. These events are undistinguishable from server side; 
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2) Activity Already Active: the user is trying to start an activity when there is an- 
other activity already active in his session; 

3) Wrong Starting Page: inside an activity, an action has been performed in a page 
that is not the last one that the user has visited; 

4) Action By completed Activity: an action has been performed within an activity 
that has been already closed. 

In the following section we will discuss all possible critical situations and exceptions 

that can be generated. 



4.2 Recovery Policies 

We define a recovery policy (for a halted activity) as a collection of operations that 
we perform on the activity and on the related data in order to bring the workflow 
application to a correct state and to let the process proceed. 

Policies can be classified with respect to three orthogonal dimensions: 

- policy direction, that considers the way in which a coherent status of the process is 
reached: the policy can try to recover a correct status that was previously visited by 
the workflow application ( backward policy), or can try to move to a new correct 
status that was not previously visited by the workflow application (forward policy). 

- policy definition, that considers who defined the policy. In this sense, we can have 
policies defined either by the workflow design framework (predefined policy) or by 
the web designer (user-defined policy, also known as compensation chain). 

- policy execution, that considers whether the policy is applied in an automated way 
(automatic policy) or in a manual way (manual policy). In the former case the policy 
is automatically applied by the workflow engine after an exception is caught and the 
engine detects a halted activity. In the latter case a user (the activity executor or an- 
other suitable user) can choose the policy to execute through a Web interface (recov- 
ery page), which is eventually reached after the activity interruption, through an ex- 
plicit login of the user (Fig. 5). 




Fig. 5. Manual policies for synchronous and asynchronous exception management 



Policies for Synchronous and Asynchronous Exceptions. Policy application can be 
affected by the type of the exception to be managed. In particular, we will apply dif- 
ferent policies depending on the fact that exceptions are synchronous or not. 




Exception Handling Within Workflow-Based Web Applications 



113 



When a synchronous exception occurs the user session is still active. To take advan- 
tage of this fact we consider only manual policy for synchronous events: when the 
exception occurs the user will be redirected to a recovery page and will choose the 
most appropriate policy(either predefined or user-defined) for the halted activity. 
When an asynchronous exception (i.e., a Session End) occurs, the user session is not 
connected any more and it is not possible to immediately apply manual policies. 
Therefore we consider both automatic and manual policy for asynchronous events. 
Automatic policies are applied automatically and transparently to the user, while 
manual policies are applied by the user itself, when he starts a new session through a 
new login. At that point the user can go in a recovery page and choose the best policy 
to apply. This behavior is depicted in Fig. 5, together with the predefined policies that 
are described in next section. 



4.3 Predefined Policies 



Our framework offers three predefined policies: Accept, Reject and Resume. To bet- 
ter understand their behavior, we consider a very simple example, consisting in the 
order payment activity, as described in Fig. 6: the activity starts, a payment is created 
and connected to the order, and the user fills up a form with his credit card data. 
Then, the payment is performed (through a black-box service) and the payment status 
is updated. If an exception occurs the current step of the activity will be step 1 (since 
it’s the only step of the activity). 





f Payment ^ 




''ModifyPayment' 1 




(End Payment) 


© 




Is®] 




• 



Payment PayActivity 



Fig. 6. Payment activity. There is just one step (the payment page), a preceding chain of op- 
eration (comprising the create payment unit and the connect unit) and a following chain (com- 
prising the unit for the payment and the modify payment) 

Accept policy. It accepts the operations already done by the halted activity, setting 
the activity status to Completed, executing all the data assignment and activating all 
the proper following activity. The process can proceed, but it may happen that part of 
the halted activity was not executed. The accept policy is a forward policy, since it 
tries to bring the workflow application to a correct status not previously visited, by 
simply assigning the status Completed to the halted activity. 

This policy is suitable only for activities that have some non critical parts, that can be 
omitted. In all the other cases, it has resulted ineffective, since it leaves the activity 
results meaningless, thus damaging the whole process case execution. For example, 
suppose that an exception occurs in the payment activity in Fig. 6. Current step is 1 , 
and if we apply an accept policy, we will consider the activity executed even if the 
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payment unit has not been performed. The process will be enabled to continue, even 
if the payment has not been performed. Therefore, in this case the accept policy is 
not a correct choice. 

Reject policy. It deletes the data created by the activity, trying to recover the initial 
state of the database before the activity execution, and assigns the Inactive status to 
the activity. The reject policy is a backward policy, since it removes the data created 
by the activity (and all the relationships with connected objects), tries to recover the 
initial state of the database before the activity execution, and assigns the Inactive 
status to the activity. Reject policy is not a full rollback mechanism, since not all the 
operations executed by the activity are undone (i.e., deletion and modification results 
are kept as they are). Indeed, we don’t want to implement a transactional system, with 
data versioning and so on. In this way, this policy can be implemented simply by 
means of a “ Created ” relationship, that connects the Activity Instance to the objects 
created in the activity itself (an example can be seen in Fig. 3). Once the reject is 
invoked, the activity is set to Inactive (ready to start) and all the Created objects are 
removed. Thus, reject is an approximate recovery of the initial state of the activity. 
This behavior partially limits the effectiveness of the policy but improves its effi- 
ciency, avoiding a performance burden resulting from a complete track of all the 
operations of the activity. Reject policy is suitable for all the activities that should be 
completely performed, and whose core task consist in creation of objects. With this 
policy, users can be asked to complete the activity repeatedly, until it is successfully 
finished. From empiric evaluations, this case results to be very frequent. 

If we go back to the payment activity example (Fig. 6), by applying the reject policy 
in case of exception, we will delete the created instance of payment entity and the 
instance of the relationship PaymentToOrder, thus canceling the effect of both the 
create payment unit and the connect unit. The activity is ready to be restarted and the 
data are in a consistent status. 

Resume policy. This policy lets the user resume browsing from the last visited page 
of the activity before the failure. This policy can be applied only by manual choice of 
the user, while the first two can be applied both automatically and manually. Brows- 
ing is resumed from the last page of the activity generated by the server, i.e. the cur- 
rent step. Note that operations with side effect are not improperly triggered by this 
policy: if the side effect occurs between the previous and the current page, it is not 
executed twice, because the user is provided with the url pointing directly to the page; 
if the side effect occurs after the current page, it has not been executed yet, otherwise 
the current page should point to the next one. 

Resume is a backward policy, since it brings the application and the workflow to a 
correct state that was previously visited. Indeed, if an exception occurs, either the user 
session is expired or the page that is shown to the user is different from the current 
step. The user cannot proceed with the execution of the correct activity, and the whole 
process status is incorrect. As we said before, the resume policy can only be applied 
through manual intervention by the user. This can be achieved by providing to the 
user a recovery page, in which he can see the activities in incorrect status, and can 
decide to resume them. By reloading the last page generated by the server on the user 
browser, the activity execution can proceed (e.g., in Payment activity example in Fig. 




Exception Handling Within Workflow-Based Web Applications 



115 



6, the resume policy lets the user reload the payment page and complete the pay- 
ment). 



4.4 Compensation Chains 

To allow a more fine-grained exception handling, we allow the designer to define his 
own recovery policies (e.g., sending warning emails to users, or implementing full 
rollback capabilities for specific activities). This solution will be adopted to manage 
the most critical activities only. The user can define operation chains that are trig- 
gered by exceptions. We provide a new unit (called CatchEvent unit), with the fol- 
lowing parameters: ExceptionType, ActivityType, Activitylnstance. Exception type 
and Activity type are specified at design time and define the situation in which the 
compensation chain is triggered. Activity instance is a runtime parameter, whose 
value is available to operations of the chain, for retrieving further related data. Note 
that, since triggering and execution of compensation chains is completely automatic, 
no pages involving user interaction are allowed within chains. Fig. 7 shows a 
sketched example of compensation chain for the Payment activity depicted in Fig. 6. 



C CatchEvent^ 


(CancPaynb 




blodifyPayment) 


(End Payment) 


f "► 

L * J 


1 © J 


b 


0 ® 
v ^ y 





PayActivity Payment PayActivity 

[status=“undone”] 



Fig. 7. Payment exception compensation chain. The payment is canceled and the information 
about it are update into the database 



5 Implementation Experience 

The concepts presented in this paper have been proved valid on the field, since a 
prototype implementation has been developed and used to design sample applica- 
tions. The implementation extends a commercial tool called WebRatio[15], which 
allows to design and automatically generate Web applications from WebML models. 
Our extension provides the workflow metadata schema and all the units presented in 
Section 2.3. Moreover, new units for granting automatic policies enactment are avail- 
able (Accept, Reject and Resume units). 

Several case studies exploiting exception handling capabilities have been imple- 
mented, thus validating and refining the approach. The results of this research, which 
is part of the WebSI project, funded by the EC’s 5 th framework, has been used by the 
partners of the WebSI project for pilot applications, and by other projects. 

Among them, Acer Business Portal (that includes remote service calls for providing 
location and driving information to users, and workflow-based interaction between 
Acer and its commercial partners), and MetalC project[ll], which is the most com- 
plex among the application we have developed, since it includes a set of B2B portals 
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(one for each business partner). The purpose of the project is to allow business inter- 
actions between Italian companies of the mechanical field by means of their respec- 
tive Web portals, through Web services calls. In this context, complex workflow 
interactions have been put in place, to grant reliable cooperation. For example, the 
purchasing process in a B2B scenario consist of a very complex set of interactions, 
since the buyer typically asks for a quote, the seller makes his offer, then the buyer 
sends his order for the best offer. In this context, exceptions management becomes 
very critic. In the implemented communication platform all the discussed recovery 
policies have been used. Some examples follows: (i) if an exception occurs within the 
AskForQuote activity, an accept policy is performed, and the request is sent even if 
not all the data are submitted (less relevant data are left in the last steps of the activ- 
ity); (ii) if an exception occurs within the SendOrder activity, the reject policy is 
applied: data created within the activity is deleted, and the user is asked to restart it; 
(iii) in case of exception within the self-registration activity, which is a long sequence 
of data submission by the partners, resume policy is exploited, to allow the user re- 
sume the self-registration from the point in which he left the application. 

An example of user defined recovery becomes necessary within the shipping confir- 
mation activity: once the order has been confirmed and the goods are ready to be 
shipped, the seller must notify the buyer about the sending. If an exception occurs 
during the execution of this activity, a user-defined compensation chain is performed, 
automatically executing the remaining steps of the activity. 



6 Conclusions 

In this paper we proposed a conceptual approach to exception handling within 
workflow-based Web applications, described through a metadata model and a set of 
primitives to be used into hypertext specification. To manage critical situations, we 
proposed an approach based on exception handling (some Java implementation al- 
ready exists that could be used to support this approach [17]), and definition of prede- 
fined and user-defined policies, that have been tested on the field. 

The main advantage of our approach stands in allowing to define exception handling 
and compensation chains without lowering the abstraction level of the design. 

Future work will address refinement of the implementation, to allow a more seamless 
and transparent integration of exception handling within WebML specification, to 
avoid the need of explicitly specifying in WebML all the basic steps of exception 
handling. A second research direction is towards study of exception handling in re- 
mote service calls. Some preliminary considerations have been done, and we expect 
an approach similar to the one we have studied for workflow-based Web applications. 
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Abstract. Loosely coupled services are gaining importance in many business do- 
mains. However, compared to OO-RPC middleware approaches, emerging tech- 
nologies proposed to implement loosely coupled services, such as Web services or 
P2P frameworks, still have some practical problems. These arise in many typical 
business domains, for instance, because of missing central control, high network 
traffics, scalability problems, performance overheads, or security issues. We pro- 
pose to use ideas from these emerging technologies in a controlled environment, 
called a federation. Each remote object (a peer) is controlled in one or more federa- 
tions, but within this environment peers can collaborate in a simple-to-use, loosely 
coupled, and ad hoc style of communication. Our design and implementation re- 
lies on popular remoting patterns. We present a generic framework architecture 
based on these patterns together with a prototype implementation. 



1 Introduction 

Loosely coupled (business) services are nowadays propagated and/or enabled by many 
different technologies, including Web services, P2P systems, coordination and coop- 
eration technologies, and spontaneous networking. Compared to OO-RPC middleware 
approaches, such as CORBA, RMI, or DCOM, these approaches promise loose coupling, 
a service-based architecture, ease of use, and ease of deployment. However, as we will 
point out in Section 2, today all these technologies have their limitations in the context of 
business-critical systems, for instance, regarding central control tasks, network traffics, 
scalability, performance, or security. 

Typical applications of loosely coupled (Web) services in different business domains 
are workflows, groupware, legacy integration, or coordination of business components. 
When we offer loosely coupled (Web) services in these business domains, there are some 
specific, recurring requirements. For instance, if spontaneous connections are allowed, 
we require some level of control to ensure that a business service cannot be misused. 
Consider an e-commerce service that should be provided only to service users who 
have paid for the service. One business peer can play more than one role in different 
contexts. Consider a peer that represents the delivery service of a content provider: it 
also has to provide a contract engine and handle rights enforcement. To model such 
situations, we cannot use the “all peers are equals” model of current P2P environments 
in the whole system. However, it would be useful, if peers that are actually equals in a 
certain situation can be handled as such. As we could use a very simple remoting model 
in such cases, this would ease the development of distributed programs significantly. 
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Moreover, service-based architectures and ad hoc connectivity may ease deployment in 
a controlled environment, say, within a company. 

We propose a federated model of remote objects as a solution. Within a federation, 
each peer offers Web services (and possibly other kinds of services) to its peers, can 
connect spontaneously to other peers (and to the federation), and is equal to its peers. 
Each remote object can potentially be part of more than one federation as a peer, and 
each peer decides which services it provides to which federation. Certain peers in a 
federation can be able to access extra services that are not offered to other peers in 
this federation via its other federations. A semantic lookup service allows for finding 
peers using metadata, exposed by the peers according to some ontology. Thus it enables 
loosely coupled services and simple self-adaptations for interface or version changes. 

We present a framework for loosely coupled Web services built internally with well 
known (OO-)RPC remoting patterns (from [21,22,23]). We will discuss a reference 
implementation written in the object-oriented Tel variant XOTcl [18] using SOAP- 
based communication. The pattern-based design has the aim that a similar framework 
can be implemented in any language with any Web service framework. The framework 
is designed to be extensible and implementation decisions, such as using a particular 
SOAP implementation as the communication protocol, can be exchanged, if required. 

In this paper, we first discuss prior work in the areas of Web services, P2P systems, 
coordination technologies, and spontaneous networking. Then we discuss open issues 
in these approaches regarding loosely coupled services. Next, we discuss a generic 
framework design and a prototype implementation in XOTcl on top of SOAP, called 
Leela. Finally, we discuss in how far the open issues are resolved in our concepts and 
conclude. 

2 Related Work 

In this section, we discuss related work in the areas of Web services, P2P systems, 
coordination technologies, and spontaneous networking. We will see that all of these 
concepts implement some of the desired functionalities, but leave a few issues open. 

2.1 Web Services 

Web service architectures center around the service concept, meaning that a service is 
seen as a (set of) component(s) together with a providing organization. Thus Web services 
are a technology offering both, concepts for deployment and providing access to business 
functions over the Web. Technically, Web services build on different Web service stacks, 
such as IBM’s WSCA [ 1 5] or Microsoft’s .NET [ 17]. These have a few standard protocols 
in common, but the Web service stack architectures are currently still diverse. At least, 
HTTP [11] is usually supported for remote communication. Asynchronous messaging 
protocols are also supported. SOAP [4] is used as a message exchange protocol on 
top of the communication protocol. Remote services can be specified with the Web 
Service Description Language (WSDL) [7], WSDL is an XML format for describing 
Web services as a set of endpoints. Operations and messages are described abstractly, 
and then bound to a concrete communication protocol and message format to define an 
endpoint. Naming and lookup is supported by UDDI [19]. 
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Each Web service can be accessed ad hoc, and services are located and bound at 
runtime. Additional composition of services is supported by business process execution 
languages, such as BPEL4WS [1], Such languages provide high-level standards for 
(hierarchical) flows of Web services. 

Web services are providing a loosely coupled service architecture and a service de- 
ployment model. However, today’s Web service stack architectures are already relatively 
complex and have a considerable overhead, especially for XML processing. Federated 
or grouped composition is not yet in focus, even though technically possible. 

2.2 Peer-to-Peer Systems 

Peer-to-Peer (P2P) computing refers to the concept of networks of equal peers collabo- 
rating for specific tasks. P2P environments allow for some kind of spontaneous or ad hoc 
networking abilities. Typical applications of P2P are file sharing, grid computing, and 
groupware. P2P computing is a special form of distributed computing that has gained 
much attention in recent times, especially P2P systems for personal use, like Gnutella, 
Napster, and others. 

Technically there are still quite diverse views on P2P. For instance, P2P can be 
interpreted as a variant of the client/server paradigm in which clients are also servers; it 
can also be interpreted as a network without servers. Often P2P is referred to as a type of 
network in which each peer has equivalent capabilities and responsibilities. This differs 
from client/server architectures, in which some computers are dedicated to serving the 
others. Note that this is only a distinction at the application level. At the technology level 
both architectures can be implemented by the same means. 

Basic functionalities shared by most current P2P systems are that they connect peers 
for a common purpose, permit users to lookup peer services, and provide a way to 
exchange data or invoke remote services. These basic properties are still quite vague - and 
do also apply for many client/server architectures. There are many optional properties, 
one can expect from a P2P system, but none is a single identifying property - that is, all 
can also be missing [8]: 

- usually there is some kind of sharing of resources or services, 

- there is an ease-of-use for users or developers, 

- there is a direct exchange between peer systems, 

- usually clients are also servers, 

- load distribution may be supported in some way, and 

- there is a notion of location unawareness regarding a used service - provided mainly 

by the lookup service used to locate a desired service. 

Regarding remote business services, P2P offers a set of potential benefits: it can be 
used to provide a very simple remoting infrastructure and loose coupling is inherently 
modeled. However, missing central coordination may cause problems regarding security, 
performance, scalability, and network traffic. 

2.3 Coordination Models 

Coordination models are foundations for a coordination language and a coordination 
system, as an implementation of the model. A coordination model can be seen as a for- 
mal framework for expressing the interaction among components in a multi-component 
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system [9]. As related work for our work, especially coordination of distributed and 
concurrent systems is of interest. The coordination language Linda [13] introduced the 
view that coordination of concurrent systems is orthogonal to the execution of operations 
(i.e. calculations) in this system. Linda can be used to model most prevalent coordina- 
tion languages. It consists of a small number of coordination primitives and a shared 
dataspace containing tuples (the tuplespace). 

The original Linda has no notions of multiple federations; a single tuplespace is used 
for all processes. However, this has its practical limitation regarding distributed systems, 
as the single tuplespace is a performance bottleneck. Moreover, there is no structuring 
of sub-spaces and scalability is limited. Bauhaus [6] introduces the idea of bags that 
nest processes in tuplespaces. A process can only be coordinated with other processes 
in the bags, or it has to move into a common bag to coordinate with other processes. 
PageSpace [10] structures Linda spaces by controlled access using different agents for 
user representation, interfaces to other agents, administrative functionality, and gateways 
to other PageSpaces. 

2.4 Spontaneous Networking 

Spontaneous networking refers to the automatic or self-adaptive integration of services 
and devices into distributed environments. New services and devices are made available 
without intervention by users. Services can be provided and located in the network. Pro- 
viding means that they can be dynamically added to or removed from the network group 
without interfering with the global functionality. Failure of any attached service does not 
further affect the functionality of the network group. Failing services are automatically 
removed and the respective services are de-registered. 

Jini [2] is a distributed computing model built for spontaneous networking. Service 
providers as well as clients firstly have to locate a lookup service. A reference to the 
lookup service can be received via a multicast. Service providers register with the lookup 
service by providing a proxy for their services as well as a set of service attributes. Each 
service receives a lease that has to be renewed from time to time. If a lease expires, the 
service is automatically removed from the network. Clients looking for a service with 
particular attributes send a request to a lookup service for such a service. In response the 
client receives all those proxies of services matching the requested service attributes. 

The Home Audio Video interoperability (HAVi) standard [14] is designed for net- 
working consumer electronics (CEs). Especially, self-management and plug & play 
functionalities are provided for spontaneous networking. Remote services are regis- 
tered using a unique Software Element Identifier (SEID) with a system-wide registry 
for service lookup. HAVi specifies the communication protocols and access methods for 
software elements in a platform-independent way. 

2.5 Open Issues for Loosely Coupled Service Architectures 

The concepts, described in the previous sections, can be used to implement loosely 
coupled service architectures. However, all have different benefits and liabilities in the 
context of business systems, such as information systems within an organization (for 
workflows, groupware, etc.) or information systems offering services to the outside 
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(such as e-commerce environments). In this paper, we propose to combine some ideas 
of these approaches to resolve the following open issues: 

- Control of Peers and Access to the Network: If a peer offers a vital service that 
should not be visible to everyone (for instance, only to those who have paid), we 
have to control access in business environments. The idea is to combine the group- 
ing concepts of coordination models, such as tuplespaces, with basic networking 
properties of the P2P model. 

- Dynamic Invocation: If static interface descriptions are mandatory for remote invo- 
cations, ad hoc connectivity is hard to model. In the context of loosely coupled Web 
services we propose the use of dynamic invocation mechanisms (on top of SOAP). 

- Simplicity: For the application developer, remoting technologies should be in first 
place simple. In a coordinated group, where we can be sure that access can be 
granted, developers of remote objects accessing the service should be able to use a 
very simple remoting model with direct interactions. 

- Security: Access to coordinated groups and the permissions what a peer can do 
within a group have to be secured. 

- Performance and Scalability: The internal protocols used should be exchangeable 
to deal with performance and scalability issues. It should be possible to replace 
performance-intensive (or memory-intensive) framework parts transparently and 
provide means for QoS control. 

- Deployment: Each accessible remote object should provide services that expose the 
ease of deployment and access known from Web services. 



3 Peer Federations 

In this section, we will step-by-step discuss our concepts for peer federations. These are 
combining concepts from the different approaches, discussed in the previous section, to 
resolve (some of) the named open issues. We illustrate our concepts with examples from 
our prototype implementation Leela. Leela is implemented in XOTcl [18], an object- 
oriented scripting language based on Tel. Our framework is designed with the remoting 
patterns 1 from [21,22,23]. We illustrate our designs with UML diagrams. 

Before we describe the peer and federation concepts, we describe the basic concepts 
of the communication framework of Leela. The communication framework’s model is 
tightly integrated with the higher-level peer and federation concepts. Therefore, it is 
important to understand its design before we go into details of the peer and federation 
concepts. 



3.1 Basic Communication Framework 

As its basic communication resource, each Leela application uses two classes imple- 
menting a CLIENT REQUEST HANDLER [23] and a SERVER REQUEST HANDLER [23] (see 
Figure 1). The client request handler pattern describes how to send requests across 
the network and receive responses in an efficient way on client side. On the server side, 

1 We highlight pattern names in smallcaps font. 
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the requests are received by a server request handler. This pattern describes how 
to efficiently receive request from the network, dispatch the requests into the server ap- 
plication, and send the response back to the client side. Each Leela application instance 
acts as a client and server at the same time. The Leela application instance and its request 
handlers can be accessed by each peer. 

The request handlers contain protocol plug-ins for different protocols that actually 
transport the message across the network. Currently, we support a SOAP [4] protocol 
plug-in. However, any other communication protocol can be used as well. As described 
below, Leela supports different invocation and activation styles (see Sections 3.4). Thus 
the specialties of most protocols supporting mainstream communication models, such 
as remote procedure calls (RPC) or messaging, can be supported. It is expected from 
the protocol that it can - at least - transport any kind of strings as message payload, 
and that one of the invocation and activation styles, supported by Leela peers, can be 
mapped to the protocol. For most protocols, it should be possible to map all invocation 
and activation styles of Leela to the protocol - of course, with different trade-offs. 




Fig. 1. Structure of the Leela Communication Framework 



Remote invocations are abstracted by the patterns requestor [23] and invoker 
[23]. A requestor is responsible for building up remote invocations at runtime and for 
handing the invocation over to the client request handler, which sends it across 
the network. The requestor offers a dynamic invocation interface, similar to those 
offered by OO-RPC middleware such as CORBA or RMI. Leela also supports peer and 
federation proxies that can act like a client proxy, offering the interfaces of a remote 
peer or federation. 

The invoker gets the invocation from the server request handler and performs 
the invocation of the peer. In Leela, there are different invokers for different activation 
strategies (see Section 3.4). The server request handler is responsible for selecting 
the correct invoker. The invoker checks whether it is possible to dispatch the invoca- 
tion; in Leela only exported objects and methods can be dispatched. This way, developers 
can ensure that no malicious invocations can be invoked remotely. 
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The Leela invocation chain on client side and server side is based on invocation 
interceptors [23]. That is, the invocation on both sides can be extended transparently 
with new behavior. Interceptors are used in Leela to add information about the Leela 
federation to the invocation (see below). Also, a client-side invocation interceptor 
can add security attributes and similar information to the invocation. A server-side in- 
terceptor can read and handle the information provided by the client. 

The requestor, invoker, and request handlers handle synchronization issues on 
client and server side. The request handlers handle the invocations according to the 
invocation and activation styles used. On server side, the server request handler 
receives network events asynchronously from a reactor [20]. The server request 
handler can have multiple different protocol plug-ins at the same time. That is, 
network events can come in from different channels concurrently. The server request 
handler queues the network events in an event loop. 

The actual invocations of peers are executed in a separate thread of control. The access 
of a particular peer can either be queued (synchronized) or handled by a multi-threaded 
object pool [23] . The results are queued again, and handed back to the receiving thread. 

On client side, different styles of asynchronous invocation and result handling are 
supported. Because in Leela each client is also a server, synchronous invocations - that 
let the client process block for the result - are not an option: if the Leela application 
blocks, it cannot service incoming requests anymore. Instead, Leela implements a variety 
of asynchronous invocation styles with a common callback model. The request handlers 
work using an event loop that queues up incoming and outgoing requests in a message 
queue. Client-side invocations run in a separate thread of control. 

The result arrives asynchronously and has to be obtained from the receiving thread. 
This is done by raising an event in the client request handler’s event loop. This event 
executes a callback specified during the invocation. An asynchronous completion 
token (ACT) [20] is used to map the result to its invocation. Using this callback model 
we can implement different asynchronous invocation styles, described in [22,23]. We 
can send the invocation and forget about the result as described by the pattern fire 
and forget, sync with server is used, when a result is not needed, but we want 
an acknowledgment from the server. Finally, the patterns poll object and result 
callback allow us to receive the result asynchronously, poll object lets the callback 
write the result to an object that is subsequently polled by the client for the result, result 
callback propagates the callback to the client - that is, it informs the client actively of 
the result. 



3.2 Invocation Types 

A remote invocation consists of a number of elements. Firstly, the actual invocation 
data consists of method name and parameters. Secondly, a service name is required - 
it is an unique object id that enables the invoker to select the peer object. Thirdly, 
protocol-specific location information is required - in the case of SOAP over HTTP this 
is the host and the port of the server request handler. The object id plus location 
information implement the pattern absolute object reference - an unique reference 
for the particular service in the network. Finally, the invocation might contain invoca- 
tion context data. The invocation context [23] contains additional parameters of 
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the invocation, such as information about the federation or security attributes. In Leela, 
the invocation context is extensible by peers and invocation interceptors. 




Fig. 2. Invocation Types and Marshallers 



Leela sends the message payload as a structured string (we use Tel lists). These strings 
are different for different invocation types. Currently, we support request, response, and 
error invocation types. The scheme is extensible with any other invocation type. The 
error message type is used to implement the pattern remoting error [23] - we use it 
to signal remoting- specific error conditions in the Leela framework. 

As shown in Figure 2 the different invocation types contain different information. 
Converting invocations to and from byte streams that can be transported across the 
network is the task of the pattern marshaller [23]. The invocation classes shown in 
Figure 2 are able to marshal and demarshal the information stored in them; thus they 
implement the main part of the marshaller pattern for the Leela framework. 

3.3 Federations and Peers 

A federation is a concept to manage remote objects in a remote object group (here 
federation members are called peers). Each federation has one central federation object 
that manages the federation data consistently. To allow peers to connect to a federation, 
the federation itself must be accessible remotely. Thus the federation itself is a special 
peer. Peers can be added and removed to a federation. 

A federation can be accessed remotely by a federation proxy. This is a special client 
proxy [23] that enables peers to access their federation, if it is not located on the same 
machine. The federation proxy is a local object that implements the federation interface. 
In principle, it sends each invocation across the network to the connected federation. 

Similar to federations, there is also a client proxy for peers, the peer proxy. The peer 
proxy basically implements the peer interface and sends all invocations to the connected 
peer of which it holds the absolute object reference. Thus, using the peer proxy, a 
local peer can interact with any remote peer that is part of its federation as if it is another 
local peer. The peer can invoke other local and remote peers using location information 
(with the method send) or using an absolute object reference (with the method 
sendAOR). The federation and peer structures are shown in Figure 3. 
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Fig. 3. Peer Federations Structure 



Federations can be introspected for their peers and properties using a semantic lookup 
service (see Section 3.6). Peers can also perform a lookup: here all federations of the 
peer are queried. 

3.4 Peer Activation 

In a loosely coupled remoting environment, activation of remote objects is a critical 
issue. Activation means creation and initialization of a remote object so that it can serve 
requests. Some peers are long living and/or persistent entities. Others are perhaps client- 
dependent such as peers that represent some session data. A client-dependent peer should 
be removed from the federation, at least, when the last peer that uses it, is destroyed or 
leaves the federation. In such cases, a lifecycle manager [23] has to ensure that these 
peers are removed from the federation, if they are not required anymore. We support the 
following activation strategies [12] (see Figure 4): 




Fig. 4. Activation Strategies Implemented on Invokers 
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- static instance: The peer is already activated before it is exported and survives 
until it is explicitly destroyed or the Leela process stops. 

- per-call instance: The class of the peer is exported and the peer is activated 
when the request arrives. Then this peer serves the request and is de-activated again. 
Per-call activated peers apply the pattern object pooling [23] - that is, they are 
pre-initialized in a pool to reduce the activation overhead for instantiation. 

- client-dependent instance: A factory operation is provided by the federation to 
create client-dependent peers, e.g. to store session data. In a remote environment, 
however, it is unclear, when a client-dependent object is not needed anymore, except 
the client explicitly destroys it. If a given object is not accessed for a while this can 
mean that the client has forgotten to clean up the instance, that it requires a longer 
computation time till the next access, that network latency causes the delay, or that 
the client application has crashed. The pattern leases [23] helps the federation to 
decide whether a certain client-dependent peer is still required. For each client- 
dependent peer a lease is started when it is created. The activating peer has to renew 
the lease from time to time. The lease is automatically renewed, when the peer is 
accessed. The client can also renew the lease explicitly using the operation ping of 
the peer. When the lease expires and it is not renewed, the client-dependent peer is 
removed from the federation. 

In Leela the lifecycle manager pattern - implementing the management of the 
activation strategies - is implemented by different elements of the framework. Peers 
are registered with an activation strategy. There are different invokers for the different 
activation strategies. The appropriate invoker is chosen by the server request 

HANDLER. 

3.5 Peer Invocation and Federation Control 

A peer may be invoked only by the federation, or by a peer in one of its federations, or 
by a local object in its own scope (e.g. a helper object the peer has created itself). Peers 
are executed in their own thread of control, and each of these threads has its own Tel 
interpreter as thread-specific storage [20]. Thus peers have no direct access to the 
main interpreter. The threaded peer interpreters are synchronized by message queues 
[23] implemented by event loops of the interpreters. That is, the peer threads can only 
post “send” and “result” events into the main interpreter - and the request handlers 
decide how to handle these events. 

In other words, each federation controls its peers. These cannot be accessed from 
outside of the federation without a permission of the federation. Of course, some peers 
in a federation need to be declared to be publicly accessible. For instance, the federation 
peer is accessible from the outside - otherwise remote peers would not be able to join 
the federation. 

Control of remote federation access is done by invocation interceptors [23] (see 
Figure 5). On client side, an invocation interceptor intercepts the construction of the 
remote invocation and adds all federation information for a peer into the invocation 
context. On server side this information is read out again by another invocation in- 
terceptor. If the remote peer is not allowed to access the invoked peer, the invocation 
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Fig. 5. Peer Federation Interceptor Structure 



interceptor stops the invocation and sends a remoting error to the client. Otherwise 
access is granted. 

Peers within a federation can access their services with equal rights. Per default each 
peer is allowed to freely send invocations to each other peer in its federation and access 
exported services. Each service offered in a federation must be explicitly exported by a 
peer. Only exported services can be accessed by other peers. By introducing invocation 
interceptors for particular peers, peer types, invokers, or requestors we can fine- 
tune the control how these elements can be accessed. For instance, we can introduce an 
interceptor that only grants access if some security credentials, such as user name and 
password, are sent with the invocation. 

Some peers are members of multiple federations. Thus they are able to access services 
of peers in other federations, something the other peers in the federation cannot do. 
Optionally, peers can act as a “bridge” to another federation - offering some of that 
federation’s services in the context of another federation. 

Figure 6 shows an example sequence diagram of an invocation sequence with a 
static invoker. Details, such as marshaling and demarshaling, are not shown here. The 
two other activation strategies just require some additional steps for interacting with the 
object pool or dealing with the lease. 



3.6 Semantic Lookup Service 

The idea to provide loosely coupled business services is often not easy to implement 
because dynamic invocation of these services requires us to know at least the object ID, 
operation name, and operation signature. To enable ad hoc connectivity this information 
can potential be unknown until runtime. Therefore, compile time interface descrip- 
tion [23] approaches (like interface description languages) are not enough. Instead a 
dynamic interface description is required as well. 

The pattern lookup [23] is implemented by many discovery or naming services. 
These provide the necessary details of any remote object that is matching a query. 
Here, often another problem is that the designers of the lookup services cannot know in 
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Fig. 6. Sequence Diagram for a Simple Invocation with a Static Invoker 



advance how the query might look like and which strategies should be applied to retrieve 
the results. For instance, always searching for all matching peers can be problematic 
regarding performance; always (deterministically) returning the first matching peer can 
cause problems regarding load balancing. Thus we propose a lookup service that is 
extensible regarding the provided information and the possible queries. 

The basic concept of the Leela lookup service is that each peer provides semantic 
metadata about itself to its federation’s lookup service. Peers can perform lookups in all 
lookup services of their federations. We use RDF [24] to describe the peers. RDF supports 
semantic metadata about Web resources described in some ontology or schema. For 
instance RDF-Schema [5] and OWL [16] support general relationships about resources, 
like “subclass of’’. Developers can also use RDF ontologies from other domains; for 
instance, in an e-learning system probably an ontology for learning materials will be 
used. 

The federation provides metadata about all its peers, such as a list of absolute 
object references and object ids (the service names). Each peer adds information 
for its exported methods, their interfaces, and their activation strategy. This information 
can be introspected by clients. 

Leela currently implements a distributed interface for the Redland RDF library [3] 
and its interface abstractions. Peers of a federation can read from and write to this 
metadata repository. As query abstractions, Redland supports the lookup of specific 
resources and sets of resources, the generation of streams, and iterators. The actual 
query is thus constructed on client side. In the future we plan to support a more powerful 
query engine on top of Redland. 



4 Discussion and Conclusion 

We have presented an approach for service-based remote programming based on remote 
object groups, called federations. The approach has similarities to Web services, P2P 
systems, coordination technologies, and spontaneous networking, but can also resolve 
some apparent open issues of these approaches. The most important design goal is ease- 
of-use regarding the development, use, and deployment of remote services. In many 
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business scenarios often a certain level of control is required. For this goal, we have 
provided a simple control model introduced by the concept of peers that canjoin multiple 
federations. Only those services that should be accessed by a remote peer are exported. 
Interceptors can be used to fine-tune the remote access. Services can be introspected for 
metadata using a semantic lookup service. Thus we can deal with unexpected lookup 
information and query types; services just have to expose additional metadata and the 
appropriate queries can be constructed on client side. 

Our design and implementation are based on well-known remoting patterns (from 
[21,22,23]) and follow the pattern language quite closely. Therefore, many underlying 
parts of our framework can be exchanged with other (OO-)RPC middleware or be imple- 
mented in other programming languages - a benefit of the pattern-based design. We are 
currently working on a Java implementation using the Apache Axis Web service frame- 
work, and we are implementing more protocol plug-ins. Thus we believe our results 
as generally applicable. Moreover, we can potentially deal with scalability and perfor- 
mance problems, as the framework is designed in such a way that the internal protocols 
and technologies are exchangeable. Our deployment model is similarly simple as Web 
service and P2P models; however, we require to know the location of at least one “well- 
known” federation to connect to a business service environment. We consider this not as 
a drawback, but an incentive in many business scenarios. Note that this “well-known” 
federation might just provide a lookup service. Thus the activities how objects are ini- 
tially located are quite similar to lookups in other middleware environments, such as 
CORBA or Web service frameworks - but they are different to those P2P environments 
that are exploiting broadcasts and similar means. 

The security aspect is handled by controlling which objects canjoin a federation and 
that only exported methods can be invoked. Each peer executes in its own interpreter 
and thread of control - thus peers cannot interfere with each other. All other security 
issues can be handled by invocation interceptors and protocol plug-ins. 

We believe our framework design to be usable in many (business) scenarios and 
plan to apply it for different applications as future work. Especially, we want to use 
the framework as a very simple remoting infrastructure. The federation model can be 
used for simple role modeling in a company; on top of such models, workflows and 
groupware applications can be implemented. As the scripting language XOTcl is pri- 
marily designed for component composition, we also want to use the Leela framework 
for distributed component gluing and coordination, especially in the context of legacy 
system integration. 

Most of the ingredients of our approach are already known from other approaches, 
but we combine them in an easy-to-use remoting concept. The framework can be ex- 
tended with add-on functionality. There are also some liabilities of the current prototype 
implementation: the current prototype does not allow for structured federations (like 
hierarchical federations), is implemented with SOAP only, and offers no QoS or failover 
control features (except those of the used Web server). We plan to deal with these issues 
in future releases of the Leela framework. 
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Abstract. Web Services and Web Service composition languages for Web 
Service choreography are becoming more and more important in the area for 
inter-enterprise application and process integration. However the aspects of 
modeling these software systems have not been studied in detail, in contrast to 
the definition of business processes where well-known techniques exist. The 
model-driven architecture (MDA) approach of the Object Management Group 
is a good starting point for the development of Web Services and Web Service 
choreography. In this paper we show how platform independent models speci- 
fied by UML 2 sequence diagrams can be automatically transformed in a Web 
Service composition language representation. 



1 Introduction 

Over the past few years, enterprises are currently in a thorough transformation proc- 
ess as they encounter the necessity to react to challenges such as globalization, unsta- 
ble varying demand, and mass customization. A most important factor to maintaining 
competitiveness is the ability of an enterprise to describe, standardize, and adapt the 
way it reacts to certain types of business events, and how it interacts as well as its 
procedures for interaction with suppliers, partners, competitors, and customers. To- 
day, virtually all larger enterprises describe these procedures and interactions in terms 
of business processes, and invest huge efforts to describe and standardize these proc- 
esses. Web Services are the key technology for Enterprise Application Integration 
(i.e. EAI; see e.g. [3] for details, [16]) and Inter Enterprise Integration. IBM defines 
Web Services as [1]: “Web se)~vices are self-contained, self-describing, modular ap- 
plications that can be published, located, and invoked across the Web (see e.g. [17]).” 
It is possible to combine them to added-value Web Services offering more function- 
ality than the original ones. This process is called Web Service Choreography or - 
composition depending on the point of view of the description. Several standards are 
under development for the definition of languages for Web Service composition or 
Web Service Choreography, typical examples are BPEL4WS [4], BPML [9], XPDL 
[13]. ebXML [8] with ebXML Business Process Specification Schema [7], 
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The Model Driven Architecture (MDA) (for details see [10, 11]) is a framework for 
software development driven by the OMG. The following models are at the core of 
the MDA: Computational Independent Model (CIM): This model describes the 
business logic and domain model; Platform Independent Model (PIM): This model 
is defined at a high level of abstraction; it is independent of any implementation tech- 
nology. Platform Specific Model (PSM): It is tailored to specify a system in terms 
of the implementation constructs available in one specific implementation technol- 
ogy, e.g. Web Services. Code: The final step in the development is the transformation 
of each PSM to code. Based on OMG’s model-driven approach, our objective is to 
demonstrate a mapping of platform independent models based on UML 2 sequence 
diagrams [12] (triggered by e.g. [15]) to a platform dependent model based on the 
Business Process Execution Language for Web Services (BPEL4WS). This paper can 
be seen as part of a series of papers dealing with software engineering starting from 
business processes and transforming them into web services choreography, see 
[19, 18]. 




Fig. 1 . Web Service enabled business processes 



2 Conceptual Methodology 

Figure 1 (an updated version of Figure 3 of [18]) illustrates the top-down develop- 
ment process starting with a semantic business process specification using and ex- 
tending UML 2.0 activity diagrams. This specification is refined into two models: a 
static model, which is essentially the service model in our conceptual methodology, 
even though enhanced with metadata, such as the description of pre- and post- 
conditions for service invocation, and with exception definitions; a dynamic model, 
which is essentially the service choreography oriented layer in the conceptual meth- 
odology. Each of these two models is described by a platform independent model and 
one or more platform specific models. In this paper we will focus on the mapping of 
UML 2 Sequence Diagrams to BPEL4WS as a part of the development process. 
Readers interested in the other parts are referred to [18, 19]. 
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3 From Sequence Diagrams to BPEL4WS 



As a next step we will now go into the details of UML 2 sequence diagrams and de- 
fine informally 1 by induction the mapping of sequence diagram elements to 
BPEL2WS, i.e. a mapping 

transform: Sequence Diagram Element — » BPEL4WS 



A sequence diagram defines an interaction denoted as 



. Thus a complete 



sequence diagram is transformed into a process definition of BPEL4WS: 



transform 



<process name = "EventOccurence" > 



transform (inner part ( 



) ) </process> 



where inner_part delivers the sub-diagram defined in the overall sequence diagram. 
Lifelines are a modeling element that represents an individual participant in an inter- 
action. A lifeline represents only one interacting entity. They are transformed by the 
following rule: 

transform ( 1 ...) = <partners> partner name = "Lifeline" serviceLinkType = 

partnerRole = myRole = </partner> ... </partners> 

Note, that the serviceLinkType, partnerRole as well as myRole are not specified in 

the sequence diagram, but have to be defined in a e.g. class diagram defining the role 

of a participant and the interface (serviceLinkType) as well as the partner role. 

Messages are translated as follows 

Synchronous/Asynchronous messages: 

sseiattu oasatigp 

Transform ( ) = creceive partner = receiver ( ) 

portType = " ... " operation = "operation" inputContainer = "operationlnC" 
outputContainer = " operationOutC" </receive> 

where receiver calculates the name of the right-hand-side lifeline name and op- 
erationlnC and operationOutC are automatically generated tokens for the 
input and output container of the operation, for aynchronous messages an output 
container is not specified since no result is transported back to the sender. 

Reply messages: 

c* o . c o 

Transform( ) = creply partner = receiver ( ) portType = "..." 

operation = "operation" container = "operationC"</reply> 

where receiver calculates the name of the left-hand-side lifeline name and op- 
erationC is an automatically generated token for the container of the operation. 



1 



Note that, a formal definition of the mapping can be based on the MOF/XMI for data ex- 
change of models; however for the sake of readability we use the graphical notation of the 
elements instead. 
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Lost and found messages are specified as usual messages with the exception of ap- 
plying the wait-construct. Co-regions are constructed with the f low-construct and the 
messages of the co-region are transformed in the usual way. 

One of the newest aspect of UML 2 sequence diagrams is the possibility to define 

«*■ ) 

combined fragements, depicted as . UML 2 distinguishes between: 

• alt - at most one of the operands will execute, this can be transformed us- 
ing the switch-construct 

» J 

transform ( ) = 

<switch> cease condition="bool-expr"> transform(operand_l) </case> 
<otherwise>? transform(operand_n+l) </otherwise> 

</switch> 

in this case the alternatives in the sequence diagram have to be annoted with 
specific conditions for each case (as in our application example in one case). 

• opt - either the (sole) operand happens or nothing happens, this is modeled 
similar to the alt-operator, where we have two cases, one case is the trans- 
formed operand and the other one is the distinguished no-operation of 
BPEL4WS. 

• loop - repeated a number of times, transformed using the while-construct 

cwhile condition= "myConstraint "> transform (operand_loop) </while> 

where myConstraint is the translated constraint of the sequence diagram. 

• par - parallel merge between the behaviors of the operands, this can easily 
be transformed with the flow-construct. 

<flow> transform(operand_l) ... transform (operand_n) </flow> 

• seq - weak sequencing depending on lifelines and operands and strict - 
strict sequencing not depending on lifelines and operands can be modeled 
with 

<sequence> transform(operand_l) ... transform (operand_n) </sequence> 

critical - critical region, this is handled by the transaction mechanism; assert - 
assertions are translated into boolean expressions which are evaluated during run- 
time; ignore - message types are not shown within fragment; consider - messages 
considered within fragment; and interaction reference - a reference to another inter- 
action, can be seen as abbreviations and need not be transformed; neg - invalid traces, 
have not been transformed. Another novelty is the usage of continuations which can 
be seen as conditional "goto" statements. These continuations can be mapped to 
BPEL4WS by applying while-loops and a boolean global variable stating if the jump 
has to performed or not. The while-statement has to be placed at the maximal com- 
prehensive block of the operands where the jump is performed. 



4 Conclusions and Outlook 

The main contribution of this paper is that it elaborates the relationship between the 
platform independent model of service choreography and its mapping to BPEL4WS a 
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specific business process execution language. The work is part of a larger project 
depicted in Figure 1. The informal definition of a mapping between the two repre- 
sentation shows that such a step can be automated. However additional information 
concerning the Web Services has to be at hand. This can be the WSDL definition of 
the Web Service interfaces specified with UML class diagrams as used e.g. by [20]. 
The next steps are the definition of a formal mapping between both representations; 
looking at a inverse mapping to allow reverse engineering; taking the "other" aspects 
of BPEL4WS into consideration, i.e. modelling of the context of the Web Service 
choreography; integration with the mapping from the computational independent 
model to the platform specific model, and integration into a development tool. 
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Abstract. In this article we present a three- level architecture for dis- 
tributed web information systems (WISs) based on media objects. We 
distinguish between a service level, a database level and an operational 
level. The media objects serve as a mediator between the service re- 
quested by users and the database. The operational level is based on com- 
municating agents that realise the functionality of a distributed database. 



1 Introduction 

Media objects [1] support tight integration between a global data and operations 
level and a local interface and dialogue level in WISs. Roughly speaking a me- 
dia object abstracts from the content and functionality of a web-page in a WIS 
including the navigation links. Using classification abstraction we define media 
types the instances of which are the media objects. The core of the definition is 
formed by a view on some underlying database schema. This view is extended 
by operations and by features that allow media objects to be tailored to different 
user needs as well as limitations arising from different end-devices or communi- 
cation channels. We present a brief overview of media types in Section 2. 

As media types provide a formal framework for WIS, the challenge in WIS 
engineering is to develop adequate system support for the storage and mainte- 
nance of media objects. In this article we address this problem and link media 
objects with a physical architecture, in which media objects can always be con- 
structed and personalised at a server receiving a request from a user. In other 
words, the media objects will act as mediators between the service level that is 
available to the WIS users and a physical database level. 

In case of database distribution we obtain a further separation between a 
global database level and an operational level, which will become available locally 
at each node of a network. Using this idea we exploit the well known approach 
of two-stack machines to realise this operational level. The major addition is 
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to extend the two-stack machines in such a way that distribution, generalised 
remote procedure calls, parallelism and communication are supported. In Section 
3 we introduce the three-level architecture for distributed WIS. 

2 Media Types 

A view V on a database schema S consists of a view schema Sy and a defining 
query gy, which transforms databases over S into databases over Sy. 

The defining query may be expressed in any suitable query language, e.g. 
query algebra, logic or an SQL-variant. For our purposes, however, this is yet 
not sufficient, since in all these cases the query result will be a set of values. One 
key concept that is missing in the views is the one of link. 

In order to introduce links, we must allow some kind of “objectification” 
of values in the query language. This means to transform a set {iq, . . . , v TO } 
of values into a set {(wi, iq), . . . , (u m , v m )} of pairs with new created URLs 
Ui of type URL. In the same way we may objectify lists, i.e. transform a list 
[vi, . . . , v m ] of values into a list [(ui, Vi), . . . , ( u m , v m )] of pairs. We shall talk of 
query languages with create- facility . This leads to the definition of raw media 
type. 

A raw media type has a name M and consists of a content data type cont(M) 
with the extension that the place of a base type may be occupied by a pair £ : M' 
with a label £ and the name M' of a raw media type, and a defining query Qm 
with create-facility such that defines a view. Here tjy is the type 

arising from cont(M) by substitution of URL for all pairs t : M' . 

Finite sets C of raw media types define content schemata. Then a database 
T> over the underlying database schema S and the defining queries determine 
finite sets T>(M) of pairs ( u , v ) with URLs u and values v of type Im for each 
M e C. We use the notion pre-site for the extension of T> to C. The pair (u,v) 
is called a raw media object in the pre-site V. 

Raw media objects are not yet sufficient for information service modelling. 
In order to allow the information content to be tailored to specific user needs 
and presentation restrictions, we must extend raw media types. 

Cohesion introduces a controlled form of information loss. Formally, we de- 
fine a partial order < on content data types, which extends subtyping [1]. If 
cont(M) is the content data type of a raw media type M and sup(cont(M)) is 
the set of all content expressions exp with cont{M ) < exp, then a partial order 
on sup(cont(M)) extending the order < on content expressions is called an 
cohesion order. Clearly, cont(M) is minimal with respect to A m ■ Small elements 
in sup(cont(AI)) with respect to define information to be kept together, if 
possible. 

Another possibility to tailor the information content of raw media types is 
to consider dimension hierarchies as in OLAP systems. Flattening of dimensions 
results in information growth, its converse in information loss. 

For a raw media type M let H(M) be the set of all raw media types arising 
from M by applying a sequence of flat-operations or their converses to raw media 
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types or underlying database types. A set of hierarchical versions of M is a finite 
subset H{M) of H{M) with M G H{M). Each cohesion order on M induces 
an cohesion order <m' on each element M' G H(M). 

A media type is a raw media type M together with an cohesion order -<m 
and a set of hierarchical versions H{M). A media schema is defined in the 
same way as a content schema replacing raw media types by media types. A 
database V over the underlying database schema S extends to a unique pre-site. 
Furthermore, we may extend T> to all hierarchical versions M' G H(M) and all 
M" >~ m> M' defined by the cohesion orders. This wide extension of V will be 
called a site. 

Finally, we collect all media types M \ , . . . , Mk together with their hierarchical 
versions and types defined by the cohesion order such that (u, G T>{Mi) holds 
for a given URL u. The pair ( u , (Mi : vi , . . . , Mk : Vk)) will be called a media 
object in the site V. 



3 The Physical Architecture 

Let us now look at the architecture that will support WIS based on media- 
objects. This architecture is illustrated in Figure 1 and we will refer to this 
figure to explain details of the architecture. 

On the top we have the service level that is formed by the provision of tailored 
media objects to various servers. The middle level is the database level , which 
is realised by a distributed database system. The lowest level is the operational 
level. The media objects are defined by views on the database, thus mediate 
between the service and the database level. 

The distributed database can be seen as a collection of physical objects stored 
on a physical storage device. These objects can be accessed through the System 
Buffer and Record Administration Server (SyBRAS). It provides efficient access 
to physical objects (synchronised by latches) and physical data independency, 
guarantees persistency for objects, etc. 

The Persistent Object Store (POS) resides on top of SyBRAS. It provides 
another level of abstraction by supporting storage objects. Storage objects are 
constructed from physical objects, e.g., records. POS maintains direct physical 
references between storage objects, and offers associative, object-related and 
navigational access to these objects. Associative access means the well-known 
access via key values. Object-related access refers to direct object access using 
object identifiers. Navigational access is related to the propagation along physical 
references [6,3]. 

Communicating agents implement the functionality of the whole system. 
They integrate the processing of queries, methods assigned to objects and trans- 
actions. Furthermore, they are responsible for distributing requests to (remote) 
agents that store corresponding objects or process requests more efficiently. 

These communicating agents are realized as two-stack abstract machines 
(2SAMs). Each agent consists of two levels. Lower levels link with POS and 
employ 2SAMs that deal with logical local objects. Higher level machines act 
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as coordinators for transactions and, thus, deal with logical global objects that 
are constructed from logical ‘local’ objects. On both levels multiple 2SAMs may 
process concurrently. In order to take advantage of concurrent processing and 
the distributed architecture 2SAMs can distribute requests to 2SAMs employed 
on the same level. However, only those machines on the higher level are equipped 
with an extended RPC enabling them to distribute requests to remote agents. 




Fig. 1 . General Architecture 



Having these multiple object levels we can take advantage of a more sophisti- 
cated transaction management system. It is based on the multi-level transaction 
model [4,2] which is counted as the most promising transaction model in theory. 
It is combined with the ARIES/ML recovery mechanism. 
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Local objects do not correspond directly to logical global objects as processed 
by the communicating agents. Furthermore, high-level queries, transactions, ob- 
ject methods, etc need to be translated into code that can be interpreted by 
communicating agents. This conceptual gap between the logical level and the 
communicating agents is bridged by a persistent reflective intermediate language 
(PRIL). 

PRIL will support linguistic reflection [5]. We provide a macro language, in 
which the high-level constructs in transactions such as generic update operations 
and the high-level algebra constructs for querying can be formulated. 

4 Conclusion 

The challenging engineering question in the area of WISs concerns system ar- 
chitectures that support WISs that are modelled as collections of media objects. 
In this article we presented a three-level architecture as an answer to this ques- 
tion. We argued that the materialisation of media objects in the underlying 
database reduces the problem to the integrated support of queries and opera- 
tions, which both arise from the definition of media types. Our architecture is 
centered around two-stack abstract machines (2SAM), but the original idea of 
such machines has been extended in a way that communication between such 
machines, parallelism and distribution are supported. This defines three-levels: 
a service level addressing the services offered to WIS users, a database level 
addressing standard schema issues, and an operational level based on 2SAM as 
communicating agents. 
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Abstract. Contract net protocol can be implemented by exploiting the Web 
Service technologies. However, the lack of a process model in contract net 
protocol to capture the interactions between the bidder and manager agents 
makes it difficult to analyze the feasibility of the resulting contracts. We pro- 
posed a framework to execute contract net protocol based on Web Services 
technologies and a Petri net model to analyze the contract net negotiation re- 
sults. 



1 Introduction 

Contract net protocol [1] is a negotiation and task distribution mechanism to optimize 
the performance in multi-agent systems (MAS) [2]. Contract net protocol can be 
implemented by exploiting the Web service discovery, invocation, selection and exe- 
cution mechanism. In this paper, we focus on modeling and analysis of contract net 
protocol that can be implemented based on Web Services technologies [3]. Given a 
set of tasks, a set of agents, a cost function, contract net protocol finds the globally 
optimal task allocation based on distributed algorithms. In contract net protocol, there 
are two types of agents: manager and bidder. Four stages are involved to establish a 
contract between a manager and the best bidders. To apply Web Services technolo- 
gies to execute contract net protocol, each manager is mapped to a Web services 
requester whereas each bidder is mapped to a Web services provider. Each bidder 
(which is a service provider) publishes the Web services by registering through a Web 
services broker. A manager (which is a service requester) looks up the registries of a 
Web service broker to discover qualified Web services provided by the potential 
bidders. Fig. 1 (a) illustrates our concept. Once the required qualified Web services 
have been found, the manager executes the contract net protocol to award the contract 
to the best bidder as shown in Fig. l(b)~(e). By combining the contract net protocol 
with Web services technologies, it is possible to automate negotiation process. 

A major drawback of the original contract net protocol is the lack of a formal 
model to capture the interactions between bidders and managers as well as the nego- 
tiation results. This makes it difficult to analyze the feasibility of the contract net 
negotiation results. For example, in real world, several contracts may need to be es- 
tablished between a company and his business partners to achieve a certain business 
objective. The lack of a model makes it difficult to efficiently evaluate the feasibility 
of the contracts. In this paper, we will propose a Petri net [4] model to model the 
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interactions between the bidders and the managers in contract net protocol and facili- 
tate the feasibility analysis of the contract net negotiation results. 




lookup 



publish publish publish 




(at 





< | > 




(b) 



(c) 




Fig. 1. Execution of contract net protocol based on Web Services technologies 



2 Petri Net Model for Evaluating Contracts 

To model the interactions between the bidders and the managers in contract net pro- 
tocol, we adopt a bottom-up approach using Petri nets, which can be broken down 
into three steps: (1) modeling of bidders’ proposals, (2) modeling the workflow of 
tasks and (3) modeling the overall system. We assume there is a set B of bidders in the 
system and each bidder be B has a set R b of type- b resources available to bid for 

several contracts. We use P h to denote the states of type- b resources and place p b0 to 
denote the idle state of type- b resources. Allocation and de-allocation of the resources 
by bidder b are represented by a circuit that starts and ends with place p b0 . The Petri 
net G b to describe a bidder’s proposal is defined as follows. 

Definition 2.1: The Petri net G b = ( P h , T h , I h , O h , m h0 , F h ) of a bid- 

der b' s proposal is a strongly connected state machine (SCSM) with any two circuits 
in G b having only one common place p b0 but having no common transitions. 

The Petri net G h = ( P b ,T b ,I b ,O b , m b0 , F b ) represents a proposal submitted by the 
bidder fee B . Fig. 2 illustrates seven bidders’ proposals using Petri nets. We assume 
that there is a set J of different types of tasks in the system. A type- j task, ye/, is 
modeled by a Petri net to executing a sequence of operations. 

Definition 2.2: The Petri net G j ■ = ( Pj , Tj , Ij , Oj , mj 0 , Fj ) of the workflow of 
a type- j task, ye / , is a strongly connected state machine (SCSM). Every circuit 
in Gj contains the idle state /; /0 , the starting transition^ , the final transition tj , the initial 

state place y? y; and the final state place p , with Pj 0 ‘ ={ tj } , ' Pjq={ tj }, tj ={ pj t } 
and 't j ={ P jj }. As each transition represents a distinct operation in a task, we as- 
sume Tj n T k = <f> for j ^ k . 
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Fig. 2. Bidders’ Proposals Fig. 3. A Task Workflow Fig. 4. Petri net Model 



Fig. 3 illustrates the Petri net model of a task workflow. Place p x represents the initial 
state place while place p & represents the final state place. All the other places in this 
Petri net represent the states of the task. Each directed path from 
place p l to p & represents an execution sequence for the task. The Petri net models for 
the bidders’ proposals and the task workflow are merged to form a Petri net Nj(nij 0 ) = 

| beB Gb | Gj = (Pj , Tj , Ij , Oj , m j0 , Fj ) to model the interactions between resources and 

a task, where || is an operator to merge a number of different Petri net models with 

common transitions, places and/or arcs.. The Petri net model in Fig. 4 is obtained by 
merging the proposals in Fig. 2 with the task in Fig. 3. 



3 Feasible Condition to Award Contracts 

The Petri net N ■ provides a model for a manager to award contracts. To characterize 
the condition to complete a type- j task, we define a feasible execution sequence as follows. 
Definition 3.1: Given a Petri net Nj with initial marking m j() , a feasible execution se- 
quence s = with t x = t'j and /j v = tj is a firing sequence such that firing s brings 

a token from the initial state place p Jt to the final state place pjj- . 

As Nj = | AeB G 6 \\Gj and Gy is a connected state machine, there may be more than one 
execution sequence for a type- j task. Plowever, an arbitrary execution sequence 
might not be a feasible one. For example, in Fig. 5, t x t 2 t 9 t w t n t n \s not a feasible exe- 
cution sequence due to the lack of a type-7 resource in place p rl . In Fig. 5, 
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t 1 t 2 t 5 t 6 t 7 t s and t 3 t 4 t 5 t 6 t 7 tg are feasible execution sequences. A feasible execution 

sequence can be characterized based on the minimal resource requirement (MRR) 
concept. 

Definition 3.2: The MRR of an execution sequence s = t l t 2 t 3 ....t<, is denoted by a 



marking m s of Nj with 

where N* s = R h © R, 2 © R t} © . . . © 




N s (b) if p 
0 otherwise 
b\ 



= p bQ for some be B 



, denotes the set of re- 



l$l 

sources required to fire s , R, , a vector in Z' , denotes the resource requirement to 
fire t only, and © is an operator that takes the larger of two vectors element by element. 
Existence of a feasible execution sequence for N f (m /U ) is as follows. 

Property 3.1: Given N,- ( m j0 ), s is a feasible execution sequence if and only if 



m j0 > m s 



Remark that Nj is constructed by merging Gj with G b for all b e B . Each execution se- 
quence of Nj must be an execution sequence of Gj . Each execution se- 
quence s = f|f 2 f 3 .../| s of Gj corresponds to a directed path y s from tj to tj in Gj . There- 
fore, to find a feasible execution sequence for a type- j task, it is sufficient to find a 
directed path y s from t' to tj with »7y 0 > m* s . 

To award the contracts for a type- j task, we find the minimum cost feasible execution 
sequence for a Petri net /V ; ( m j() ) . The problem is formulated as follows. Let c, denote 
the cost to fire a transition t , where te T j . The cost to fire the se- 
quence s of transitions is then c(s) = ^ c, . Let Sj denote the set of all 

tss 



feasible execution sequences of Nj(m j0 ) . The optimization problem can be formu- 
lated as the problem to fmdminc(,s). The problem to findminc(s)is equivalent to 

se Sj se Sj 

solve min c(y) , where Y-J eas,ble ={ yjy is a directed path from t’j to and ot / 0 > m* } 

y E Y-f eaSible 

denotes the set of all feasible directed paths in Nj . 

Definition 3.3: LotTf easible = { t\te Tj,R, < m j0 } and Tf easib,e = Tj\ T( easible be the 
set of transitions that cannot be fired due to insufficient resource tokens in m /0 . 

The problem min c(y ) can be converted to a shortest path problem by constructing a 

y E Y-f eaSible 

weighted directed graph Gj based on Gj and Tj eas,b,e ; where G can be constructed by 
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removing p J0 and the set Tj easMe { f , tj } of transitions from Gy , and then convert- 
ing each transition in the resulting graph into a directed arc and assigning a weight to it. The 
weight w t of a transition t isc, for eachte jf eas,ble The shortest path problem described 

by Gy can be solved by applying the Dijkstra’s algorithm with polynomial complexity. 
Fig. 6 illustrates the Petri net Gy obtained based on Gy in Fig. 3 and 
T" 1 easib,e = { t 10 , t n } . Existence of a shortest path depends on Gy and marking m J0 . 




4 Conclusion 

We focused on modeling and analysis of contract net. We identified a feasible condi- 
tion to award contracts and formulated an problem to find a minimal cost feasible 
execution sequence for a task based on Petri net. The optimization problem can be 
converted to a shortest path problem, which can be solved efficiently. 



References 

1. R.G. Smith, “The Contract Net Protocol: High-Level Communication and Control in a 
Distributed Problem Solver”, IEEE Trans. On Computers Vol. 29, pp. 1104-1113, 1980. 

2. Jacques Ferber, “Multi-Agent Systems, An Introduction to Distributed Artificial Intelli- 
gence,” Addison Wesley, 1999. 

3. S. Mcllraith, T.C. Son, H. Zeng: Semantic Web Services, IEEE Intelligent Systems, Special 
Issue on the Semantic Web, 16 (2) (2001) 46-53. 

4. Tadao Murata, ”Petri Nets: Properties, Analysis and Applications,” Proceedings of the 
IEEE , vol. 77, No. 4, April 1989. 




A Web Metrics Survey Using WQM 



Coral Calero, Julian Ruiz, and Mario Piattini 
ALARCOS Research Group 

Computer Science Department. University of Castilla-La Mancha 
Paseo de la Universidad, 4 
13071, Ciudad Real (Spain) 

{Coral . Calero , Julian. Ruiz, Mario . Piattini } @uclm. es 



Abstract. Quality is an essential characteristic for web success. Several authors 
have described different methodologies, guidelines, techniques and tools in 
order to assure the quality of web sites. Recently, a wide ranging set of metrics 
has been proposed for quantifying web quality attributes. However, there is 
little consensus among them. These metrics are sometimes not well defined, nor 
empirically or theoretically validated. Moreover, these metrics focus on 
different aspects of web sites or different quality characteristics, confusing, 
rather than helping, the practitioners interested in using them. With the aim of 
making their use easier, we have developed the WQM model (Web Quality 
Model), which distinguishes three dimensions related to web features, lifecycle 
processes and quality characteristics. In this paper we classify the most relevant 
web metrics using this framework. As a result of this classification we obtain 
that most of the metrics are classified into the “usability / exploitation / 
presentation” cell. Another conclusion obtained from our study is that, in 
general, metrics are automated but not validated formally nor empirically which 
is not a good way of doing things. 



1 Introduction 

Nowadays web technology is of paramount importance in Information Systems. In 
fact, the world economy’s slowdown has not affected the web field because large 
firms stopped expanding, and began consolidating and moving to the web, to cut costs 
[45], Over the next few years, the web is expected to increase by a factor of 20, 
growing to 200 million sites by 2005, and the number of actual web pages will 
increase even more [42]. 

The ever increasing presence of web technology and its importance for the survival 
of organizations make it essential to develop a complete Web Information Systems 
Engineering (WISE), meant as a collection of sound principles, methods, techniques 
and tools for developing web-based information systems, which differ from 
traditional information systems in their unique technological platform and design 
philosophy [37] and quality assurance is one of the challenging processes to the Web 
Engineering as a new discipline [11]. WISE aims improve and achieve quality web 
sites. Despite discussion of sticky web sites and development of mechanisms to 
encourage users to return, thus far the only mechanism that brings repeat users to web 
sites is quality [36]. 
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However, and perhaps because the quality of web sites is not universally definable 
and measurable [9] their quality is not always assured [2, 10]. 

In recent years several experts have worked on different proposals to improve web 
quality: methodologies [39], quality frameworks [13, 25], estimation models [28], 
criteria [50], usability guidelines [34], assessment methods [49] and metrics. 

In fact, web metrics is a particularly valuable area of ongoing commercially 
relevant research [47]. Since the nineties, a wide ranging set of metrics has been 
proposed for quantifying web quality attributes [1, 3, 5-7, 12, 14, 17, 18, 22-24, 26- 
31,33, 38-41,44-46,51]. 

However, these metrics are sometimes not well defined and neither empirically nor 
theoretically validated. All these metrics, focused on different aspects of web sites or 
different quality characteristics, can confuse, rather than help, the practitioners 
interested in using them. 

With the aim of classifying these metrics and making their use easier, we have 
elaborated the WQM model (Web Quality Model), which distinguishes three 
dimensions related to web features, lifecycle processes and quality characteristics 
[48], 

Recently, Dhyani et al. [12] proposed a web classification framework using 
different categories: web graph properties, web page significance, usage 

characterization, web page similarity, web page search and retrieval, and theoretical 
information. The authors try to determine how the classified metrics can be applied to 
improving web information access and use. However they discard other important 
dimensions such as lifecycle and web features which are included in our model. 
Moreover in this survey they do not consider some very interesting metrics such as 
[24, 28,38]. 

In the following section we present the WQM model explaining each of its 
dimensions. In the third section we will summarize the result of the classification of 
the most relevant web metrics. Conclusions and future work will appear in the last 
section. 



2 Dimensions in Web Quality 

In Ramler et al. [43] the authors define a cube structure in which they consider three 
basic aspects when making a test of a web site. Following this idea, in Ruiz et al. [48] 
we proposed another “cube” in which the three dimensions represent those aspects 
that must be considered in the evaluation of the quality of a web site: features, life 
cycle processes and quality aspects, which can be considered orthogonal. We have 
used this model to classify different studies on web engineering and we have refined 
our dimensions. In this section we will summarize the last version of the WQM, 
which is represented in figure 1 . 



2.1 Web Features Dimension 

In this dimension we include the three “classic” web aspects: Content , Presentation 
and Navigation [6, 15, 16]. Navigation is an important design element, allowing users 
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Fig. 1. Graphic representation of the model. 



to acquire more of the information they are seeking and making that information 
easier to find. Presentation and content are prime components in making the page 
easier to use [42]. 

In Content we have included not only data such as text, figures, images, video 
clips, etc, but also programs and applications that provide functionalities like scripts, 
CGI programs, java programs, and others. Content also deals with structure and 
representation issues. Due to the close intertwining of functions and data the border 
between them is not clearly drawn, and we consider them the same feature. 

Navigation concerns the facilities for accessing information and for moving around 
the web. 

Presentation is related to the way in which content and navigation are presented to 
the user. 



2.2 Quality Characteristics Dimension 

For the description of this dimension we use as a basis the Quint2 model [35] based 
on the ISO 9126 standard [20]. We have decided to use Quint2 instead of the standard 
because this model extends the ISO standard with new characteristics very 
appropriate for web products. Quint2 is a hierarchical model that fixes six basic 
characteristics, each one of them with a set of subcharacteristics, to which a set of 
attributes is associated. These are the basic elements. Table 1 shows the 
characteristics of Quint2, indicating, if necessary, those subcharacteristics added or 
removed respect to ISO 9126. 

There is also a compliance subcharacteristic for all characteristics (attributes of 
software that make it adhere to application related standards, conventions in laws and 
similar prescriptions). 
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Table 1. Model Quality Characteristics 

Functionality. A set of attributes that bear on the existence of a set of functions and their specified properties. The 
functions are those that satisfy stated or implied needs. 

■ Suitability: Attribute of software that bears on the presence and appropriateness of a set of functions for specified tasks. 

■ Accuracy : Attributes of software that bear on the provision of right or agreed results or effects. 

■ Interoperability: Attributes of software that bear on its ability to interact with specified systems. 

■ Security: Attributes of software that bear on its ability to prevent unauthorized access, whether accidental or deliberate, to programs 
or data. 

■ Traceability (Quint2): Attributes of software that bear on the effort needed to verify correctness of data processing on required 
points. 

Reliability. A set of attributes that bear on the capability of software to maintain its level of performance under 
stated conditions for a stated period of time. 

■ Maturity: Attributes of software that bear on the frequency of failure by faults in the software. 

■ Fault tolerance: Attributes of software that bear on its ability to maintain a specified level of performance in cases of software 
faults or of infringements of its specified interface. 

■ Recoverability: Attributes of software that bear on the capability to re-establish its level of performances and recover the data 
directly affected in case of a failure and on the time and effort needed for it. 

■ Availability (Quint2): Attributes of software that bear on the amount of time the product is available to the user at the time it is 
needed. 

■ Degradability (Quint2): Attributes of software that bear on the effort needed to re-establish the essential functionality after a 

breakdown. 

Usability. A set of attributes that bear on the effort needed for use, and on the individual assessment of such use, by 
a stated or implied set of users. 

■ Understandahility: Attributes of software that bear on the users’ effort for recognising the logical concept and its applicability. 

■ Learnahility: Attributes of software that bear on the users’ effort for learning its application (for example, control, input, output). 

■ Operability: Attributes of software that bear on the users’ effort for operation and operation control. 

■ Explicitness (Quint2): Attributes of software that bear on the software product with regard to its status (progression bars, etc.). 

■ Attractivity (. Attractiveness in Quint2): Attributes of software that bear on the satisfaction of latent user desires and preferences, 
through services, behaviour and presentation beyond actual demand. 

■ Customisability (Quint2): Attributes of software that enable the software to be customized by the user to reduce the effort required 
for use and increase satisfaction with the software. 

■ Clarity (Quint2): Attributes of software that bear on the clarity of making the user aware of the functions it can perform. 

■ Helpfulness (Quint2): Attributes of software that bear on the availability of instructions for the user on how to interact with it. 

■ User-friendliness (Quint2): Attributes of software that bear on the users’ satisfaction. 

Efficiency. A set of attributes that bear on the relationship between the level of performance of the software and the 
amount of resources used, under stated conditions. 

■ Time behaviour: Attributes of software that bear on response and processing times and on throughput rates in performing its 
function. 

■ Resource behaviour: Attributes of software that bear on the amount of resources used and the duration of such use in performing its 
function. 

Portability. A set of attributes that bear on the ability of the software to be transformed from one environment to 
another. 

■ Adaptability: Attributes of software that bear on the opportunity for its adaptation to different specified environments without 
applying other actions or means than those provided for this purpose for the software in question. 

■ Installability: Attributes of software that bear on the effort needed to install the software in a specified environment. 

■ Replaceability: Attributes of software that bear on the opportunity and effort of using it in the place of specified other software in 
the environment of that software. 

■ Co-existence (not included in Quint2): The capability of the software to co-exist with other independent software in a common 

Maintainability. A set of attributes that bear on the effort needed to make specified modifications. 

■ Analysability: Attributes of software that bear on the effort needed for diagnosis of deficiencies or causes of failures, or for 
identification of parts to be modified. 

■ Changeability: Attributes of software that bear on the effort needed for modification, fault removal or for environmental change. 

■ Stability: Attributes of software that bear on the risk of unexpected effect of modifications. 

■ Testability: Attributes of software that bear on the effort needed for validating the (modified) software. 

■ Manageability (Quint2): Attributes of software that bear on the effort needed to (re)establish its running status. 

■ Reusability (Quint2): Attributes of software that bear on its potential for complete or partial reuse in another software product. 
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2.3 Life Cycle Processes Dimension 

By introducing this dimension, we believe that we are also considering the people 
involved in the development who have different skills and therefore different 
priorities and attitudes [32] are included. For example, the developer’s interests are 
considered in the development process. 

So, in this dimension we include the diverse processes of the web site life cycle 
following the ISO 12207-1 standard [19]. In the current version of the model we only 
included three main processes: the development process, the exploitation process 
(which includes the operative support for users) and the maintenance process (which 
includes the evolution that the web site undergoes). 

It is important to emphasize that the activities of these processes must not be 
developed sequentially, because, due to the characteristics of web development, it will 
be necessary to use more iterative models and even more flexible developments 
without following formal methodologies [4]. 



3 Analysis of Existing Metrics 

3.1 Surveyed Metrics 

For the present study, we have surveyed different studies of metrics related in some 
manner with web topics. We have reviewed about 60 papers, from 1992 to 2003. 
From all these we have selected the ones (about 40) where metric proposals 
(considered useful for classification purposes on WQM) were included, discarding 
some others where the proposed metrics were not really applicable in our context or 
did not provide any relevant information. Examples of the discarded metrics include 
all the process metrics, focusing our work only on product metrics. We also discarded 
repeated metrics, i.e., those metrics proposed by more than one author. 

We included each metric only once. 326 metrics were selected, and are listed at the 
end of this paper. Finally, we wish to note that the process of classifying metrics is 
not a simple task and we are conscious that some of the assignments may be arguable. 



3.2 Filling the Cells of the Cube 

Although the model does not restrict the number of cells that can be assigned to a 
given metric m, for the sake of simplicity and practicality we tried to minimize this 
number by assigning the metrics to the cells where they could be most useful. To 
avoid unnecessary complexity, we decided to show in the WQM only the quality 
characteristic assigned, instead of the precise sub-characteristic. 

Assigning metrics to life cycle processes was not easy. We have given some 
special consideration to exploitation and maintenance. In the web world, where 
typical timeline in web development is 3-6 months [44], it is difficult to distinguish 
when exploitation finishes and maintenance begins. In case of doubt we have 
classified metrics in both processes. 
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3.3 The Resulting Cube 

Due to the extent of the detailed assignments of metrics to cells, this information is 
included at the end of this paper. 

In this section we will summarize the main figures of our classification shown in 
table 2. The “% Absolute” row shows the percentage of metrics classified on each 
value dimension and the sum of these values is greater than 100% because, as we 
have already explained, a metric can be classified in more then one cell in the cube. 
Because of this we have extracted prorated values shown in the “% Prorated” row. 



Table 2. Metrics Classification. 





Web Features 


Quality Characteristics 


Lifecycle 

Processes 


Con- 

tent 


Presen- 

tation 


Naviga- 

tion 


Functio- 

nality 


Relia- 

bility 


Usa- 

bility 


Effici- 

ency 


Porta- 

bility 


Main- 

tain- 

ability 


Develop- 

ment 


Exploi 

tation 


Mainte- 

nance 


Total 


99 


179 


67 


50 


21 


263 


47 


40 


79 


64 


267 


162 


% Absolute 


30% 


55% 


21% 


15% 


6% 


81% 


14% 


12% 


24% 


20% 


82% 


50% 


% Prorated 


29% 


52% 


19% 


10% 


4% 


53% 


9% 


8% 


16% 


13% 


54% 


33% 



Figure 2 shows metric distribution over the three dimensions of the model: web 
features, quality characteristics, and lifecycle processes, using prorated figures. The 
next subsections present several conclusions that we can extract from it. 



3.3.1 Web Features Dimension 

About 52% of the metrics were “presentation” metrics. This value confirms the 
tendency in the web world to give it the greatest importance, making the sites as 
attractive as possible for the end user. 

At this point it is convenient to remark that usually there is a confusion between 
presentation and navigation [6] so, perhaps the results of the navigation could vary 
depending on the person who makes the classification. 



3.3.2 Quality Characteristics Dimension 

Most of the metrics (53%) are usability metrics. We have to take into account that this 
data is prorated, because if we examine absolute data (table 2) we can see that 81% of 
metrics are related to usability. Again this value confirms the end-user focus trying to 
design usable web sites that attract users. 

However, it is curious that only 4% of metrics focus on reliability, when this 
characteristic it is also extremely important for customer acceptance of web sites. 

Finally, we think that the appearance of new devices (such as PDA, mobiles, ...) 
will encourage the definition of new portability metrics. 



3.3.3 Life-Cycle Dimension 

With respect to life cycle, the exploitation and maintenance processes are the ones 
with most metrics. These results can be justified by taking into account the 
evolutionary nature of the web. The fact that there are not too many metrics defined 
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Fig. 2. Metric Distribution across the Model Dimensions 



for the development process can be explained because getting their software to the 
market first is the top priority for firms doing business on the web and so, rather than 
develop software from requirements through the waterfall, web developments firms 
try to use rapid application development methods and continuous prototyping [44] . 



3.4 Metrics Properties 

We have also evaluated the metrics considering the following properties [8]: 

• Granularity Level, depending on whether the metric focuses on a single 
web page or on a web site. 

• Theoretical Validation helps us to know when and how to apply metrics. 

• Empirical Validation, with the objective of proving the practical utility of 
the proposed metrics. 

• Automated Support, i.e., whether or not there is a support tool that 
facilitates the calculation of the metrics. 

The results of this evaluation are shown at the end of this document. 

As we can see there is a balanced distribution of metrics defined for web pages 
(47%) and web sites (53%). 
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The results of the validation confirm that, unfortunately, web metric validation is 
not considered as a major issue, especially theoretical validation (4%) but also, 
empirical validation (32%). 

A large number of metrics are automated (79%). This is very important if we want 
to incorporate the metrics into web development and maintenance projects. 



4 Conclusions and Future Work 

There are many metric proposals for web quality, but no consensus has been reached 
for their classification. To advance in this area, it is essential to rely on a model that 
allows us to classify and systematize metric use. In this paper we have presented the 
WQM and we have surveyed the most relevant web metrics. 

Nevertheless, this is only a first approach that needs to be reviewed until a 
definitive and complete version is reached that can be used with total reliability and 
guarantee of success. 

Regarding the model, some modifications could be carried out in the life cycle 
dimension including a project process (following the standard ISO 15288, System 
Life Cycle Processes [21] in order to include in the WQM proposals related to web 
estimation effort like Mendes et al. [28-31]. We think this point is particularly 
interesting because as remarked in Reifer [44] web developments are hard to estimate 
and many professionals try to avoid this difficulty by using the more traditional 
processes, metrics and models for estimating web projects. However, these traditional 
approaches do not seem to address the challenges facing the field. 

It could also be interesting to consider the metrics related to cost estimation 
because this is an essential element for providing competitive bids and remaining 
successful in the market [47]. 

Regarding the metrics classified in this study, we do not claim this survey to be 
complete. It would be necessary to make an even more exhaustive study of the state 
of the art. We also intend to define new metrics in those “cells” in which the 
nonexistence of metrics is detected. 
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Abstract. In the paper we address the problem of estimating the effort required 
to develop dynamic web applications. In particular, we provide an adaptation of 
the Cosmic Full Function Point method to be applied on design documents for 
counting data movements. We also describe the empirical analysis carried out 
to verify the usefulness of the method for predicting web application develop- 
ment effort. 



1 Introduction 

In the present paper we address the problem of estimating the effort required to de- 
velop web applications. This represents an emerging issue in the field of web engi- 
neering, due to the dramatic increasing of complexity and size of web applications, 
and the consequent demand for tools supporting project development planning with 
reliable cost and effort estimations [2,6,7,8,9,10,11]. 

In [9] Rollo employed COSMIC-FFP ( Cosmic Full Function Point) [5], an adap- 
tation of Function Points [1] originally defined for real-time applications, to measure 
the functional size of an Internet bank system. Following his suggestion in [6] Men- 
des et al. provided a formal method which adopts COSMIC-FFP to measure size of 
static hypermedia web applications. In the paper we propose to apply the COSMIC- 
FFP method also to dynamic applications. Indeed, since COSMIC-FFP measure is 
focused on the counting of data movements, it turns out to be suitable for client- 
server applications, which are characterized by large amounts of data movements. To 
provide an early size estimation, we propose to apply the method on design docu- 
ments. In particular, we extend the proposal by Mendes et al., by defining some rules 
that allow us to measure functional size of client-server applications, using class dia- 
grams. Moreover, we also report on an initial empirical validation of the approach 
performed on a set of dynamic applications developed by undergraduate students of 
two academic courses on web engineering. 

The paper is organized as follows. In Section 2 we describe the rules to adapt 
COSMIC-FFP counting to class diagrams describing dynamic web applications. Sec- 
tion 3 presents the results of the empirical analysis carried out so far. Section 4 con- 
cludes the paper giving some final remarks and discussion on future work. 
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2 Applying COSMIC-FFP to Dynamic Hypermedia Applications 

The COSMIC-FFP method is focused on counting data movements from Functional 
User Requirements to obtain the functional size of the software, expressed in terms of 
CFSU ( Cosmic Functional Size Unit) [5]. Each CFSU corresponds to either an entry, 
or an exit , or a read, or a write. 

In the sequel, suitable rules are given for counting data movements of dynamic 
web applications using class diagrams, expressed in terms of the UML notation for 
the web proposed in [3], 

1 . For each static web page count 1 entry + 1 read + 1 exit. Indeed, an entry is sent to 
the application by requesting the client page {entry), the page is read from the web 
server ( read) and then shown to the user (exit). 

2. For each multimedia component, which is visualized after an explicit request of the 
client, count C*(l entry + 1 read + 1 exit). C denotes a weight associated to the 
component and is determined by considering its influence on the development pro- 
cess. In particular, C=l, means little influence; C=2, means medium influence; 
C=3, means strong influence. 

3. For each script used to provide a functionality to manipulate document on the client 
side (i.e., in the web browser), count 1 entry. 

4. For each application executed on the client side, count C*(l entry + 1 exit), where 
C denotes again a weight. The entry is considered to run it and the exit to show it. 

5. For each server side interpreted script or compiled module used to produce a dy- 
namic web page, count 1 entry + 1 read + 1 exit. In this case, a form allows users to 
input data and request a dynamic page (entry). The web server elaborates the input 
of the user through the server-side script or module (read) and produces a web page 
which is sent to the user (exit). Moreover, count an additional read if an access 
control is first performed and then the input is elaborated to generate the web page. 

6. For each server side script modifying persistent data through the web server, count 
1 entry + 1 write + 1 exit. The user inputs data through a form (entry), the data is 
written through the web server (write) and the result is shown to the user (exit). 
Count an additional read if an access control is first performed. 

7. For each web page that contains confirmation, alert or error messages sent by the 
web server to the browser, count 1 read + 1 exit. 

8. For each reference to external applications such as commercial package, library 
routine, count 1 entry + 1 exit. 

Let us note that rules 5, 6, 7, 8 were specifically conceived to consider dynamic 
aspects of web applications, rule 2 refers to multimedia components and rules 1, 3, 4 
take into account elements common to static web applications. In particular, the latter 
rules are analogous to the ones provided by Mendes et al. in [6] to measure hyperme- 
dia web applications. The sum of the identified data movements is expressed in terms 
of CFSU. 

In the sequel we show the application of some rules. To this aim, let us consider 
the class diagram depicted in Fig. 1, which is referred to a web application designed 
for e-learning purposes and is concerned with the final test. The user requests the 
final test by specifying his/her data through the HTML form UserRegistration con- 
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tained in the client page FincilTest. The server page Userldentification verifies 
whether or not the user is registered and the server page TestCrecition prepares the 
form TestForm by using the information of the class Test. The user fills in the form 
by answering the questions and submits his/her test. Then, the server page Scoring 
interacts with the database and determines the score which is sent back to the user as 
an HTML page (i.e., Score). Moreover, the server page DBUpdating inserts the score 
into the database by using the user data contained in the object Session. The presence 
of the server pages Userldentification, TestCreation, Scoring determines the applica- 
tion of rule 5, resulting in 9 CFSUs. Rule 6 is instead applied considering the server 
page DBUpdating, determining other 3 CFSUs. Finally, the presence of the static web 
page FinalTest which contains the HTML form UserRegistration, causes the applica- 
tion of rule 1, counting further 3 CFSUs. Thus, the total counting for the considered 
piece of design documentation is 15 CFSUs. 




Scoring ' 



STUDENT 
(^>LastName 
if^FirstName 
^>ID 
i^Score 

♦lnsertScore() 

♦DeleteScoreQ 



\«Redirect» 



© 



DBUpdating 



Session 



(^LastName 

i^FirstName 

§>\D 



Fig. 1 . The UML class diagram modelling the final test within an e-learning course 



3 Empirical Evaluation 

A statistical analysis has been performed to establish whether the proposed applica- 
tion of COSMIC-FFP can be used to predict the development effort of web based 
systems. We have exploited data coming from 32 web projects developed by students 
during the course on Web Engineering of two subsequent academic years. In both 
cases, the most skillful students were equally distributed among the groups in order to 
allow uniformity. Table 1 provides the descriptive statistics performed both for the 
variable Effort ( EFH ), expressed in terms of person-hours, and the variable COSMIC- 
FFP ( C-FFP ), expressed in terms of CFSUs. 
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Table 1 . Descriptive statistics of EFH and C-FFP 





Obs 


MIN 


MAX 


MEAN 


STD. DEV. 


EFH 


32 


62 


172 


117.6250 


33.6699 


C-FFP 


32 


84 


833 


346.4688 


225.6862 



In order to perform the empirical validation of the proposed method, we have ap- 
plied an Ordinary Least-Squares regression analysis. The scatter plot of Fig. 2. a ex- 
poses a positive linear relationship between the variables EFH and C-FFP. From the 
Q-Q plot of Fig. 2.b we can observe that the residuals are normally distributed. 



Normal Q-Q Plot of Studentized Residual 




Fig. 2. (a)The scatter plot with EFH on the y-axis and C-FFP on the v-axis. (b) The Q-Q plot 
of the residuals. 



From Fig. 3. a, we can observe that the linear regression analysis shows a high ad- 
justed R~= 0.763, which indicates that 76.3% is the amount of the variance of the de- 
pendent variable EFH that is explained by the model. Moreover, we can observe a 
high F value and a low p-value, which indicate that the prediction is indeed possible 
with a high degree of confidence. The p-values and 1-values for the corresponding 
coefficient and the intercept are reported in Fig. 3.b. The p-values give an insight into 
the accuracy of the coefficient and the intercept, whereas their t-values allow us to 
evaluate their importance for the generated model. 



Prediction Model 


adjusted R 2 


R 


Std Err 


F 


p-value 


EFH=0. 13 1 *C-FFP +72.254 


0.763 


0.878 


16.3978 


100.699 


0.000 



(a) 





Value 


Std. Err 


t- value 


p-value 


Coefficient 


0.131 


0.013 


10.035 


0.000 


Intercept 


72.254 


5.371 


13.453 


0.000 



(b) 



Fig. 3. The results of the OLS regression analysis for evaluating the EFH using C-FFP 

In order to assess the acceptability of the derived effort prediction model, we have 
considered the Mean of Magnitude of Relative Error (MMRE) [4] over the 32 obser- 
vations. The model exhibits an MMRE = 0.1151, which is less than 0.25, the thresh- 
old suggested by Conte et al. for an effort prediction model [4]. Moreover, we have 
evaluated the prediction at level 0.25 [4], which has turned out to be 0.8438. Ac- 
cording to Conte et al. a good effort prediction model should exhibit a value not less 
than 0.75, which is the case for the derived model. 
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4 Final Remarks 

In this paper we have proposed an approach for estimating the functional size of dy- 
namic web applications, exploiting the COSMIC-FFP method, during design phase. 
The results of the empirical analysis that we have carried out are encouraging, and 
suggest several research directions for future work. First of all, further analysis is 
planned for the assessment of the method. Indeed, the empirical evaluation provided 
in the paper has to be considered a preliminary analysis since the use of students’ 
projects may threat the external validity of the experiment and hence data coming 
from the industrial world are presently being collected, in order to obtain more reli- 
able results. Such data will be also used to perform a comparative analysis with re- 
spect to other proposals, such as Web Objects [10,11], and to refine the current pro- 
posal by considering other features, such as the complexity of the data structure and 
algorithm behind the data movement. We finally plan to automate the application of 
the proposed rules on the class diagrams in order to integrate such functionality in a 
suitable CASE tool. 
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Abstract. Over the past five years a small number of specific commercial 
processes and evolutions to traditional software engineering processes for Web 
Engineering have been proposed. The existing Web engineering literature 
focuses mainly on techniques and tools that underpin the process of building 
Web applications, with little or no focus on the commercial suitability of the 
Web Engineering processes themselves. Based on our experience and surveys 
of Web engineering in practice, we have defined a set of essential criteria to be 
addressed by a commercial Web engineering process. In this paper we present a 
systematic evaluation of a sample of commercial Web engineering processes 
against these criteria. None of the commercial Web engineering processes 
evaluated addresses all the identified criteria. Ultimately to address the criteria 
for a Web engineering process there is a need for a different type of process. 



1 Introduction 

Our experience and that of others suggest that Web engineering requires different 
software development processes from traditional software engineering. From October 
until December 2000 we carried out a survey of commercial organisations involved in 
Web-based application development [1, 2], Based on our personal experience, this 
survey and other surveys of Web engineering in practice, we have identified seven 
characteristics of Web engineering that we believe must be addressed by a Web 
engineering processes. These are support for: 

1 . Short development life-cycle times 

2. Different business models (Business Process Re-engineering) 

3. Multidisciplinary development teams 

4. Small development teams working in parallel on similar tasks 

5. Business Analysis and Evaluation with End-Users 

6. Explicit Requirements and rigorous Testing against requirements 

7. Maintenance 

Koch [3] in her comparative study paper identified several differences between the 
development of hypermedia systems and traditional software development. These 
included: different developer skills, taking into account aesthetic and cognitive 
aspects; augmented user role; and the more important role of the maintenance phase. 
She concluded "in particular, some research is needed to improve and test methods 
that cover the complete life cycle of hypermedia applications” and argued for better 
requirements capture and more focus on validation, verification and testing. 
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2 Evaluating Commercial Web Engineering Processes 



Over the past five years a small number of specific processes and evolutions to 
traditional software engineering processes for Web Engineering have been proposed. 
Processes specifically for Web engineering include Collaborative Web Development 
[4] and Crystal Orange Web [5]. Evolutions to traditional software engineering 
process include extensions to OPEN [6] and to The Rational Unified Process [7]. 

This section provides a short introduction to each process, followed by a table 
describing our analysis of how each of the processes supports our criteria for Web 
engineering processes. Each process evaluated is given a rank against each of the 
criteria, points 1-7, listed above, indicating how strongly a particular process supports 
each criterion under the following scheme: none, weak, partial, strong or very strong. 

The Collaborative Web Development (CWD) process [4] is based on Burdman’s 
extensive experience of developing Web-based applications. However, she comes 
from a technical and creative writing background and this is strongly reflected in the 
process she describes. The CWD process life-cycle is plan-driven, with four phases: 
‘Strategy’; ‘Design and Specification’; ‘Production’; and ‘Testing and Launch’. The 
communications model proposed is very hierarchical and appears rather heavyweight 
for Web-based applications. Table 1 describes our analysis of CWD’s support for the 
criteria for a Web Engineering Process. 

Table 1 . Collaborative Web Development’s support for the Criteria for a Web Engineering 
Process 



No. 


Support 


Comments 


1 . 


Partial 


CWD prohibits any changes being incorporated during the production phase 
without cost implications. This approach is only suitable within contracting 
projects with fixed requirements. 


2. 


None 


No support for impact into business and domain models from the software 
model. 


3. 


Partial 


CWD only supports the creative design and development roles, ignoring the 
business and domain experts. 


4. 


Weak 


Team structure is the same regardless of the size of the project and there is no 
inter-team communication model for parallel development. 


5. 


Weak 


There is some support for business analysis in the Strategy phase but a lack of 
explicit focus on Evaluation within CWD. 


6. 


Weak 


The Design and Specification phase, combining requirements and design to 
produce a technical specification in CWD is known to be problematic [1, 2]. 
Conventional testing by development team only. 


7. 


Partial 


CWD maintenance focus is on software and creative design models, ignores 
business and domain models. 



Crystal Orange Web (COW) [5] is part of a family of agile processes developed by 
Alistair Cockburn. Crystal processes are: “people- and communication-centric”; 
intended for development teams that are collocated within the one building and are 
not designed for safety critical systems. Crystal methodologies should be adjusted to 
fit a particular setting and Crystal Orange Web is an application of the Crystal 
methodology used to deliver a Web-based application for eBucks.com. Table 2 
describes our analysis of COW’s support for our criteria. 
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Table 2. Crystal Orange Web support for the Criteria for a Web Engineering Process 



No. 


Support 


Comments 


1 . 


Strong 


COW recommends two-week development cycles, with a further 
recommendation that “each team must deliver something useful to the public 
every four weeks”. 


2. 


Weak 


COW encourages “business owners” to consider the business processes 
required for software failure or errors. However there is no explicit 
encouragement to discuss the potential benefits that may be derived by re- 
engineering business processes. 


3. 


Strong 


One of the objectives of COW is that ideally, the programmers, user interface 
designers, testers, business owners, marketing experts, et al. should sit in 
cross-functional teams. 


4. 


Weak 


There is an absence of support for concurrent development in COW, although 
other Crystal processes show strong support for this criterion. 


5. 


Weak 


Business analysis involves the business owner writing a business use case and 
a system use case brief. COW does not mention end-user involvement during 
the development life-cycle before live delivery of software. 


6. 


Partial 


Requirements are supported by business analysts producing detailed use cases 
and data descriptions. COW emphasises support for testing by the developers. 


7. 


Partial 


There is no mention of long term maintenance and evolution issues within 
COW, the focus is on rapid short-term evolutionary steps. 



Web OPEN is based on Object-oriented Process, Environment and Notation (OPEN) 
[8], an object-oriented process framework developed and maintained by over thirty 
five members of the OPEN Consortium. A recent paper [6] describes how OPEN can 
be extended to “fully support the new demands of website construction and the 
delivery of business value on the Web”. 

In assessing the suitability of Web OPEN we have considered the explicit extensions 
to the basic process [6] and also where the extended process depends upon the 
foundations of the OPEN process framework. Table 3 describes our analysis of Web 
OPEN against our criteria. 

There are extensions for Web application development with IBM’s Rational Unified 
Process (RUP) [9], a well known plan-driven software process product which is 
widely used for the development of object-oriented systems. A Rational white paper 
[7] describes how the RUP can be extended for Web-based application development. 
This paper ‘ focuses particularly on the front-end of the lifecycle, and how to integrate 
the creative design process with the software engineering process of the Rational 
Unified Process”. 

In assessing the suitability of the extended form of RUP we have considered the 
explicit extensions to the basic process [7] and also where the extended process 
depends upon the foundations of the RUP process. Table 4 describes our analysis of 
how the proposed extensions to RUP support the criteria for a Web Engineering 
Process. 



3 Conclusions 



Using the criteria for the evaluation of Web engineering processes we have evaluated 
a number of commercial Web engineering processes using the available literature. 
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Table 3. Web OPEN support for the Criteria for a Web Engineering Process 



No. 


Support 


Comments 


1 . 


Weak 


The OPEN framework requires process engineers to create a process instance 
particular to their project or organization. There is therefore a strong 
dependency on a skilled process engineer, Cockburn level 3 [5], to ensure that 
the process is sufficiently tailored to deal with the time-to-market pressures 
experienced in Web engineering. It is unlikely that most Web projects will 
have access to such a skilled process engineer. 


2. 


Weak 


The Web OPEN process does not address impact from software model back 
into the business and domain models nor does it not mention business process 
re-engineering. The OPEN process framework does however provide a Phase 
within the Enterprise Lifecycle known as Business Reengineering. 


3. 


Partial 


Web OPEN includes the creative design and developer roles but there is no 
mention of the business or domain expert roles. 


4. 


None 


There is no mention of how teams are coordinated or work together within a 
large Web engineering project. 


5. 


Partial 


The OPEN framework includes business modelling but it is not clear how this 
relates to business analysis. Web OPEN does not mention an Evaluation 
phase or the involvement of end-users during development. However, OPEN 
provides a number of work products for Usability Testing, although we could 
find no discussion of where, how or when these should be applied in Web 
development. 


6. 


Strong 


OPEN provides a focus on requirements engineering and Web OPEN 
provides specific focus on testing through the Web Site Testing Task. 


7. 


Strong 


There is explicit focus in Web OPEN on a new Activity known as Web Site 
Management dealing with the bringing “ together of all the issues regarding 
the development, maintenance and management of a corporate website which 
may or may not include access to back-end transaction processing systems ” 



Table 4. RUP and extensions support for the Criteria for a Web Engineering Process 



No. 


Support 


Comments 


1 . 


Weak 


Process is too predictive and heavy weight, requiring the development of a 
‘Full Web User Interface prototype’, (six documented deliverables), to be 
produced covering all use-cases, before the construction phase. Recommends 
re-use of use cases from previous Web projects to address time-to-market. 


2. 


None 


The impact reflected within the process is from the business and domain 
model to the software and creative design models, as opposed to just the 
software model as in vanilla RUP. There is no mention of impact back into 
the business and domain models from the software model. 


3. 


Partial 


Focus on the increased visibility of the creative design role when building 
Web solutions. There is explicit mention of a wider diversity of stakeholders 
required to build Web solutions than in traditional software engineering, but 
no integration of these roles into the development process. 


4. 


Weak 


No mention of parallel activities or coordinating many small teams. 


5. 


Partial 


RUP includes business modelling workflow before deriving software 
requirements. No explicit mention of evaluation with end-users or how to 
support this activity within the process. 


6. 


Strong 


Explicit focus on capture of all types of requirements including functional 
(use case model) and non-functional, including those specifically relevant to 
the creative design model. The testing element is contained within vanilla 
RUP but there is no explicit mention of Web site testing. 


7. 


Partial 


No explicit mention of maintenance issues. However a number of documented 
deliverables are produced within the creative design and software models. 



Our analysis shows that Crystal Orange Web is the only process that explicitly 
addresses the crucial criterion of short development life cycles. There is clearly a need 
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for stronger support for different business models and business process re- 
engineering, reflecting impact back from the software model into the business and 
domain models. COW is the only process to incorporate the wide range of 
development roles required in Web engineering. None of the processes provide a 
mechanism to support scalability to a number of small teams working in parallel. 
There is a need for stronger focus on addressing the customer community view (end- 
users and those impacted by the project deliverables) within commercial Web 
engineering processes and particularly end-user participation throughout development 
and evaluation. With respect to requirements, testing and maintenance the extensions 
to traditional software engineering processes provide stronger support because of their 
software engineering process foundations. 

Our original motivation for doing this work was to identify criteria for Web 
engineering processes that would underpin our research in developing a new Web 
engineering process. Our work on the Agile Web Engineering (AWE) process has 
been described elsewhere [10]. However, we believe that these criteria for evaluating 
Web engineering processes are much more widely applicable. Further empirical 
evidence will either strengthen or modify our assessment of the essential criteria, 
which can then be used to evaluate other Web engineering processes, both 
commercial processes and those proposed by other researchers in the field. They can 
also be used as the starting point for further research into Web engineering processes. 
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Abstract. The webLyzard project generates empirical Web data by processing 
large samples of Web sites automatically. It mirrors more than 5,000 interna- 
tional Web sites in monthly intervals and has amassed Web data in excess of 
one terabyte since 1999. Structural and textual analyses convert the wealth of 
information contained in the sample into detailed site profiles and aggregated 
content representations. A distributed approach promises to increase both sam- 
ple size and the frequency of data gathering. This paper presents a roadmap to- 
wards distributed Web assessment, extending and revising the current system 
architecture to enhance its scalability and flexibility for investigating the dy- 
namics of electronic content. 



1 Introduction and Methodology 

Software agents extracting and processing Web sites automatically ensure scalability, 
speed, consistency, rigorous structure, and abundant longitudinal data. They alleviate 
methodological limitations of subjective impressions and anecdotal evidence [1-3]. 
Judged against human evaluation, automated approaches are more efficient at han- 
dling dynamic Web data, and immune to inter- and intra-personal variances. These 
advantages come at the expense of sacrificing recipient-dependent attributes, which 
are difficult to quantify. Domain knowledge and expert opinions, therefore, play an 
important role when interpreting and applying the results. 

This paper presents a roadmap towards distributed Web monitoring in terms of 
analytical objectives, process and data structures, and interface representation. Higher 
demands on transparency, flexibility and portability require new structures for data 
representation and transformation based on XML schemas and XSLT stylesheets, 
respectively. These data structures will pave the way for advanced visualisations. 
Topographic information mapping and geo-referenced projections will encourage the 
interactive exploration of Web resources. 



2 Methodology 

The webLyzard project has built the foundation for regularly sampling thousands of 
Web sites [2, 3]. As the volume and constantly changing char- acter of Web data 
entails ongoing analysis, a crawling agent mirrors the Web sites in monthly intervals, 
aptures their characteristics and stores the resulting profiles in a relational database. 
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The move from centralised to distributed data gathering will not only increase the 
frequency of data gathering (weekly or daily intervals, depending on the site’s char- 
acteristics), but also put samples of 100,000 sites and more within reach by leverag- 
ing spare bandwidth and previously unused computing resources from geographically 
dispersed clients. 

The methodology considers both visible (raw text including headings, menus, or 
link descriptions) and invisible text (embedded markup tags, scripting elements, and 
so forth). Ignoring graphics and multimedia files, the agent follows a site’s hierarchi- 
cal structure until reaching 10 megabytes for regular sites, or 50 megabytes for online 
media. The size restriction helps compare systems of heterogeneous size, and manage 
storage capacity. Documents found in lower hierarchical levels are not part of the 
main user interface and can be disregarded for most analytical objectives. 

2.1 Sample specification includes adding new Web sites, updating external rankings, 
and regularly checking the validity of addresses. The current sample of more than 
5,000 Web sites comprises the Fortune 1000 (www.fortune.com), international media 
sites, environmental non-profit and advocacy organisations, European tourism sites, 
and nominees of the 2003 Webby Awards. 

2.2 Data extraction processes and automatically codes the mirrored data. Automated 
coding attenuates subjective interpretations and many of the questionable aspects of 
manual coding. It renders the problems of time-consuming and expensive coder 
training or intra- and intercoder reliability obsolete, as the process measures variables 
directly and does not involve individuals who could disagree on particular attribute 
values. The variables constituting the site profiles fall into one of four categories: 
navigational mechanisms, interactive features, layout and multimedia characteristics, 
and linguistic descriptives. 

2.3 Data representation of the current system needs a fundamental revision in order 
to meet the requirements of distributed Web monitoring (see Section 2.8). Currently, 
the variables are stored in a relational database and exported as comma-separated text 
files for further processing by external applications. The revision aims at replacing 
the current structures with a modular and reusable repository of Web metrics based 
on XML technologies. The flexibility of data definitions via XML schemas will pro- 
vide the foundation for the envisioned distributed architecture. XSLT stylesheets will 
transform structural site metrics into diverse output formats such as XML, X(HTML), 
.CSV, and .PDF. The portability of XML-encoded data will encourage collaborative 
development of analytical modules. 

2.4 Structural analysis relies upon multivariate statistical techniques or supervised 
neural networks to determine success factors by correlating the site profiles (set of 
independent variables) with measures of online success such as network statistics, 
server log-files, and questionnaire data (dependent variables). Important aspects of 
the proposed research are evaluating the availability and suitability of external rank- 
ings, and utilising third-party data such as Google Page Ranks (www.google.com), 
Alexa traffic statistics (www.alexa.com), and organisational details from Internet 
registrars (www.internic.net). The increasing availability of Application Program- 
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ming Interfaces (APIs) and specialised scripting languages such as the Compaq Web 
Language [4] and the HTML Extraction Language [5] should facilitate these tasks. 

2.5 Textual analysis converts the raw data into adequate representations, assigns 
languages to each document, and eliminates ambiguities in order to provide linguistic 
site metrics and identify focal points of environmental Web coverage. Most methods 
for analysing textual Web data originate from corpus linguistics and textual statistics 
[6, 7]. The type token ratio, for example, indicates the richness of the vocabulary of a 
given text by dividing the number of distinct words (types) by the total number of 
words (tokens) encountered when processing the text file. 

The study of multicultural Web samples requires explicit attention to the role of 
language. Trigrams (three letter sequences within a document) and short words (de- 
terminers, conjunctions or prepositions) are good clues for guessing a language. As 
the results of both methods are nearly identical for textual segments comprising more 
than ten words [8], the computationally less intensive short word technique has been 
chosen for the current prototype. The proposed research aims to evaluate algorithms 
for document classification and employ more advanced parsing of sentence structures 
to eliminate ambiguities concerning the syntactic function or semantic nature of 
words, and explore knowledge representations such as XML topic maps [9] to handle 
dynamic and often redundant content segments. 

2.6 Topic detection and tracking identifies semantic clusters in continuous streams 
of raw data that correspond to previously unidentified events [10]. Web-based track- 
ing of events and public opinion on environmental issues complements and enhances 
traditional questionnaire surveys. Presenting news items culled from approximately 
4,500 sources worldwide, Google News (news.google.com) demonstrates that auto- 
matic content classification by computer algorithms without human intervention can 
produce viable results. Standardised document type definitions - e.g., the News 
Markup Language (www.newsml.org), which is used by major international content 
providers such as Reuters or United Press International - facilitate classifying and 
aggregating content from multiple sources. 

2.7 Visualisations of environmental Web content incorporates knowledge from 
cartography and geo-informatics [11], allowing visual recognition of document simi- 
larity by spatial cues and increasing the accessibility of complex data sets. The project 
will evaluate visualisation techniques and automate the creation of perceptual maps, 
currently performed manually by post-processing the output of statistical standard 
software. Scalable Vector Graphics (SVG), a modularised language for describing 
two-dimensional vectorial graphics in XML syntax [12], will be used to generate on- 
the-fly representations of Web content including frequency, context, geographic dis- 
tribution, and hierarchical position of characteristic terms. 

2.8 Distributed Web monitoring applies emerging computing frameworks that have 
revolutionised the processing of complex information. As some simulations are be- 
yond the reach of current supercomputers, Seti@Home (setiathome.berkeley.edu), 
Climateprediction.net or LifeMapper.org exploit spare computer cycles by breaking 
down the calculations into parallel units that are processed by networks of volunteers 
(Seti@home scans radio signals from space to search for extraterrestrial intelligence; 
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Climateprediction.net explores climate change caused by manmade atmospheric pol- 
lutants; Lifemapper maps ecological profiles based on records of the world's natural 
history museums). As participating in active research attracts a large number of users, 
these projects often scale to thousands or millions of nodes. With the support of 
nearly 4.8 million users, Seti@Home executes more than 60 trillion floating point 
operations per second (TeraFLOPS) as of January 2004. This surpasses the capacity 
of the world’s fastest supercomputer - i.e., the Earth Simulator, a Japanese climate 
modelling project generating a maximum of 36 TeraFLOPS (www.es.jamstec.go.jp). 

There are two main approaches to the distributed processing of information. Peer- 
to-Peer (P2P) computing has been popularised by file sharing and the highly parallel 
computing applications described above. Grid computing, by contrast, serves smaller 
communities and emphasises large-scale systems with fixed or slowly changing node 
populations in environments of at least limited trust [13, 14]. 

Migrating from centralised to distributed system architectures is complex and la- 
bour-intensive. Therefore, the project will utilise a standard service layer to collabo- 
rate with other projects, streamline system implementation, and manage computing 
resources more effectively. Examples of P2P service layers are the Berkeley Open 
Infrastructure for Network Computing (boinc.berkeley.edu), XtremWeb (www.lri.fr/ 
-fedak/XtremWeb), and JXTA (www.jxta.org). The Open Grid Services Architecture 
(www.globus.org/ogsa) is based on Web service technologies to provide a more 
service-oriented platform, allowing computational services providers to register 
within known registries that can be browsed and searched by clients [15]. This 
achieves a cleaner middleware layer for individual applications based on industry 
standards such as the Web Services Description Language (WSDL) and the Universal 
Description Discovery and Integration (UDDI) protocol (www.w3.org/TR/wsdl; 
www.uddi.org). 

While the project will benefit from the scalability and fault tolerance of P2P com- 
puting, its data-intensity suggests exploring light-weight Grid frameworks - as exem- 
plified by the Knowledge Grid [16], a knowledge extraction service on top of the 
Globus (www.globus.org) grid architecture. 

Distributed Web crawling permits gathering data in weekly or daily intervals, de- 
pending upon the media’s dynamic characteristics. Leveraging spare bandwidth and 
previously unused computing resources from geographically dispersed clients, sam- 
ples of 100,000 sites and more become feasible, limited only by the number of par- 
ticipating individuals and institutions. As this process consumes lots of bandwidth but 
demands less resources in terms of processing capacity, it complements computation- 
intensive projects such as Seti@home and Climateprediction.net. 



3 Conclusion 

The roadmap presented in this paper incorporates knowledge of multiple disciplines 
to better understand the determinants, structure and success factors of Web content. 
The project builds upon the technology of the webLyzard project, which analyses 
more than 5,000 international Web sites in monthly intervals. Distributed data gath- 
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ering will allow running the crawling agent in weekly or daily intervals. Leveraging 
spare bandwidth and previously unused computing resources from geographically 
dispersed clients, samples of 100,000 sites and more will become feasible, limited 
only by the number of participating individuals and institutions. 

The use of open standards for encoding both structural and textual site data will 
enable other disciplines and stakeholders to effectively leverage the gathered infor- 
mation, which will be available via the collaborative environment of the Web portal 
(www.ecomonitor.net). A combination of the content management platform’s core 
components and customised modules will facilitate the exchange of information with 
the research community, international and regional media, and the general public. 
This ensures a broad and international base of active researchers who are likely to 
contribute computational resources to the data gathering process, access and use the 
provided Web services, and promote the project among their peers. 
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Abstract. There are various useful ISO standards related to software quality 
models, measurement, and evaluation processes. However, we observe 
sometimes a lack of a sound consensus among the same terms in different 
documents or sometimes absent terms. In [7, 10], our ontology for software 
metrics and indicators was detailed. In this article, the process and decisions 
that have been taken during the construction of the ontology are highlighted. 



1 Introduction 

Software measurement and even more web measurement are currently in a stage in 
which terminology, principles, models, and methods are still being defined and 
consolidated. In order to promote a more mature discipline, it has becomes mandatory 
to start reaching a common agreement between researchers and other stakeholders 
about primitive terms such as attribute, metric, measure, measurement and calculation 
method, scale, elementary and global indicator, calculable concept, among others. 

One key factor for the success of our proposed cataloging web system [9] was the 
conceptualisation of the software metrics and indicators domain formalized by an 
ontology. Then, our aim was to construct an explicit specification of the 
conceptualisation for this domain, where concepts, attributes, and their relationships 
were clearly represented. In this paper, the process and decisions that have been taken 
during the construction of the ontology are highlighted. We explain why the meaning 
from one proposal and not from the others was chosen, or why new terms were 
needed. For this end, the main steps for building the ontology using the Methontology 
strategy [1] is presented in Section 2. In Section 3, a discussion about decisions taken 
in choosing the terms is presented. Finally, concluding remarks are drawn. 



2 Methodology for Building the Ontology 

A considerable number of methodologies to build ontologies have been published so 
far in which different design principles and stages for ontology development were 
reported. One of the well-known methods is the Methontology strategy [1], It 
proposes an effective, generally applicable method for domain knowledge model 
construction and validation as well. At least, it distinguishes the following steps: 
Specification, Conceptualisation, Implementation, and Evaluation. 
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Specification Step. In a general sense, a sound metrics and indicators specification, 
flexible documentation, consultation, and retrieval mechanisms are needed in order to 
contribute to the comprehension and selection process whether metrics and indicators 
can be useful, easy to collect, and understand. For this end, we argued that a well- 
designed repository of metrics and indicators and a semantic web-based cataloging 
system [9] can be effectively used to support quality assurance processes such as 
nonfunctional requirement definition and specification, metrics and indicators 
understanding and selection, amongst others. Therefore having an explicit ontology 
was a key requirement for our cataloging system. 

On the other hand, the sources of knowledge for the proposed metrics and 
indicators ontology came from our own experience backed up by previous works in 
metrics and evaluation processes and methods, from different software-related ISO 
standards, and also from recognized research articles. Taking into account some of his 
own previous works, Olsina [8] authored the Web Quality Evaluation Method 
(WebQEM), which is grounded on the design, selection and implementation of 
metrics, elementary and global indicators and their associated decision criteria, 
starting from a calculable concept (e.g. quality), and its model. A further research was 
aimed to specify web metrics, and to develop a conceptual model just for metrics and 
its cataloging system [9]. Mainly, our ontology has also been inspired in sources as 
ISO standards and other recognized research articles and books. To quote just a few: 
The terminology provided in the ISO 15939 standard [5], as well as in the ISO 9126-1 
[4], and in the ISO/IEC 14598-1 standards [3], 

Conceptualisation Step . Conceptualisation helps to organize and structure the 
acquired knowledge using external representation languages (such as UML diagrams, 
tables, classification trees, etc.) that are independent of implementation languages and 
environments. As results of this step, the metrics and indicators knowledge using a 
UML conceptual model was yielded, where the main domain concepts, attributes, and 
relationships were represented (see details in [7, 10]). In table 1 the glossary of terms 
for the metrics and indicators ontology is explicitaly described. 

Implementation Step. To implement the ontology in a formal language, we mainly 
used RDF/S, since it is the associated language for ontology implementation with 
semantic web capability. Implementing the metrics and indicators ontology with 
semantic web technologies provides more interoperability, automateability, and 
scalability to our cataloging web system, as discussed elsewhere [9, 10]. 



3 Discussion About Decisions Taken in Choosing Some Terms 

It is worthy of mention that there are various useful recently issued ISO standards 
related to software quality models [4], measurement [5], and evaluation processes [3]. 
The primary aim of these standards was to reach a consensus about the issued models 
or processes together with a consensus about the used terms; however, they do not 
constitute themselves a formal or a semiformal ontology. 

The ontology for software metrics and indicators we are discussing was based as 
much as possible on the defined terms of the ISO standards. Specifically, eight terms 
and their exact meaning out of twenty seven terms of our proposal were fairly used, 
namely: the attribute [3], decision criteria [5], entity [5], information need [5], 
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measurable concept [5] (we used the “calculable” word instead of “measurable” as we 
discuss later on), measure [3], scale [3], and unit [5] terms. In addition, we employed 
almost the same meaning to the metric term as in [3], i.e., “the defined measurement 
and calculation method and the measurement scale”. We argue that a direct metric 
uses just a measurement method meanwhile an indirect metric can use both 
measurement and calculation methods -an indirect metric is represented by a function 
or formula that specifies how to combine metrics. 

Considering these ISO standards, we have very often observed a lack of sound 
consensus among the same terms in different documents or sometimes absent terms. 
For instance, the “metric” term is just used in [3, 4] but not in [5]. Even more, [3, 4] 
use the terms “direct measure” and “indirect measure” (instead of direct or indirect 
metric), meanwhile [5] uses “base measure” and “derived measure”. In some cases we 
could state that they are synonymous terms, but in other such as metric, which is 
defined in [3] as “the defined measurement method and the measurement scale”, there 
is no term with exact matching meaning in [5]. Furthermore, we argue that the 
measure term is not synonymous with the metric term. The measure term defined in 
[3] (the meaning we adopted) as “the number or category assigned to an attribute of 
an entity by making a measurement” or in [5] as “variable to which a value is 
assigned as the result of measurement” reflects the fact of the measure as the resulting 
value or output for the measurement activity (or process). Thus, we claim the metric 
concept represents the specific and explicit definition of the measurement activity. 

Besides, we observe some absent terms in these standards such as “elementary 
indicator” and “global indicator” (even though in [5] the “indicator” term is defined 
with similar but not equal meaning as in our proposal), as well as the “calculation” and 
“calculation method” terms that are totally absent. For us, the intended objective for 
using a measurement method is slightly different for that of using a calculation 
method. The former is just intended for a measurement activity; the latter, just for a 
calculation activity. 

Focusing us again on the metric term, the metric m represents the mapping m:A -> 
X, where A is an empirical attribute of one or more entities (the empirical world), X 
the variable to which categorical or numerical values can be assigned (the formal 
world), and the arrow denotes a mapping. In order to perform this mapping a sound 
and precise measurement (activity) definition is needed by specifying explicitly the 
method and scale. 

On the other hand, a semantic distinction between metric and indicator concepts 
should be raised. The indicator represents a new mapping coming from the 
interpretation of the metric’s value (formal world) into the new variable to which 
categorical or numerical values can be assigned (the new formal world). In order to do 
this mapping a model and decision criteria for a specific user information need is 
considered. It is interesting to observe the definition of the “rating” term in [3] that 
says “the action of mapping the measured value to the appropriate rating level” in 
addition to the “indicator” term in the same document that says “a measure that can be 
used to estimate or predict another measure”. However our meaning for the indicator 
term (see table 1) is broader in the sense of an explicit definition of the calculation 
activity needed to produce an indicator value. 
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Table 1 . Software metrics and indicators ontology: Glossary of Concepts. 



Concept Name 


Description 


Attribute 


A measurable physical or abstract property of an entity [3]. 


Calculable 

Concept 


Abstract relationship between attributes of entities and information needs [5]. 


Calculation 


Activity that uses an indicator definition in order to produce an indicator’s value. 


Calculation 

Method 


The particular logical sequence of operations specified for allowing the 
realisation of a formula or indicator description by a calculation. 


Categorical Scale 


A scale where the measured or calculated values are categories, and cannot be 
expressed in units, in a strict sense. 


Concept Model 


The set of sub-concepts and the relationships between them, which provide the 
basis for specifying the concept requirement and its further evaluation or 
estimation. 


Decision 

Criteria 


Thresholds, targets, or patterns used to determine the need for action or further 
investigation, or to describe the level of confidence in a given result [5]. 


Direct Metric 


A metric of an attribute that does not depend upon a metric of any other attribute. 


Elementary 

Indicator 


An indicator that does not depend upon other indicators to evaluate or estimate a 
calculable concept. 


Elementary 

Model 


Algorithm or function with associated decision criteria that model an elementary 
indicator. 


Entity 


Object that is to be characterised by measuring its attributes [5]. 


Function 


Algorithm or formula performed to combine two or more metrics. 


Global Indicator 


An indicator that is derived from other indicators to evaluate or estimate a 
calculable concept. 


Global Model 


Algorithm or function with associated decision criteria that model a global 
indicator. 


Indicator 


The defined calculation method and scale in addition to the model and decision 
criteria in order to provide an estimate or evaluation of a calculable concept with 
respect to defined information needs. 


Indicator Value 


The number or category assigned to a calculable concept by making a 
calculation. 


Indirect Metric 


A metric of an attribute that is derived from metrics of one or more other 
attributes. 


Information Need 


Insight necessary to manage objectives, goals, risks, and problems [5]. 


Measure 


The number or category assigned to an attribute of an entity by making a 
measurement [3]. 


Measurement 


Activity that uses a metric definition in order to produce a measure’s value. 


Measurement 

Method 


The particular logical sequence of operations and possible heuristics specified for 
allowing the realisation of a metric description by a measurement. 


Method 


Logical sequence of operations and possible heuristics, specified generically, for 
allowing the realisation of an activity description. 


Metric 


The defined measurement or calculation method and the measurement scale. 


Numerical Scale 


A scale where the measured or calculated values are numbers that can be 
expressed in units, in a strict sense. 


Scale 


A set of values with defined properties [3]. 


Software Tool 


It is a tool that automates partially or totally a measurement or calculation 
method. 


Unit 


Particular quantity defined and adopted by convention, with which other 
quantities of the same kind are compared in order to express their magnitude 
relative to that quantity [5]. 



In addition, we would like to remark the introduction of the terms categorical and 
numerical scale to our ontology (that are not explicitly specified in the ISO standard). 
This distinction is important to understand the unit concept associated to scales (see 
definitions in table 1). Particularly, a categorical scale is a scale where the measured or 





180 L. Olsina and M. de los Angeles Martin 



calculated values are categories, and cannot be expressed in units, in a strict sense. 
Instead, in a numerical scale the values are numbers that must be expressed in units. 

Lastly, at the moment of publishing our ontology the closest related work was the 
recent proposal of the software measurement ontology documented by Genero et al. 
[2], which was inspired mainly in the terms of the ISO 15939 standard. This work had 
been a consultation source for our proposal, nevertheless, we have aimed mainly to 
the metrics and indicators ontology rather than to the measurement process ontology. 
Our ontology could serve as a subontology for that. In the same direction, with the 
aim to reach a consensus in a measurement ontology, we were maintaining 
discussions with different Ibero-American research groups, where a final technical 
report containing the glossary of terms will be published in 2004 (we held three 
physical meetings in 2003). As results of the joint discussions we have not yet reach a 
total agreement in all the terms so far. The main discrepancies are in the metric and 
indicator terms. Some participants claim that the definition of the metric term is as 
follows: “a defined measurement form -i.e., a measurement method, or a function, or 
an analysis model-, and a scale in order to perform measurements of one or more 
attributes”. Moreover, an indicator is a kind of metric (an inheritance relationship). 
For instance, under this consideration (likewise in [2]), the double mapping for an 
indicator is not clearly represented. Despite these enriching discrepancies a final joint 
report will be issued, which could be referenced as another source of knowledge. 



4 Concluding Remarks 

As Kitchenham et al. say [6], without sound and agreed definitions it is difficult to 
assure metadata consistency and, ultimately, data values are comparable on the same 
basis. We hope our proposal will have a good diffusion within the software and web 
communities, and also can serve as a trigger for new enriching discussions as well. 
The stability and maturity of an ontology can also be judged by the level of agreement 
reached in a domain-specific international community. This involves the evolveability 
and perfectibility features of any consensuated knowledge building process. 
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Abstract. Our contribution in this paper is an approach to measure 
semantical relations within a web site. We start with a web page de- 
scription by key words. The implementation of structural and content 
information reduces the variety of key words. Thereby, the document- 
key-word-matrix is smoothend and similarities between web pages are 
emphasized. This increases the possibility of cluster key words and iden- 
tif topics successfully. To do so, we implement a probabilistic clustering 
algorithm. To assess semantic relations, we introduce a number of mea- 
sures and interpret them. 

1 Introduction 

Facing a huge, complex corporate web site, it is difficult to keep track of daily 
changing web pages and their relationship to the rest of the web site. Therefore, 
it is helpful to provide an insight into the content structure of a web site. With 
this semantic information it would be possible to improve the web site design 
and its usability. 

In order to create a semantical overview of a web site, the first step should 
be the analysis of the web content. In a second step we should analyze the struc- 
ture of the web site. Based on the assumption that the link structure provides 
useful semantic clues [2], it is important to integrate these implicit information. 
Additional insight into the semantical structure can be provided if each page 
is integrated into its context [3]: If a page is not regarded as a solitaire, the 
information of its neighborhood is relevant for its understanding. 

In contrast to this approach, related work, done in this area of researclr[2] 
uses a pre-defined set of topics. Clrakrabati et al. [2] argues that predefined topic 
taxonomies circumvent key word ambiguity. For their purpose, it is reasonable 
since they do not focus on huge corporate web sites. In contrast we do and 
aim to create a generic solution. Like them, we do not rely on well-kept meta 
information. Including an identification of hubs and authorities [1] has to be 
analyzed whether it can improve our results. Reasonable results of document 
clustering incorporating hyperlink structures can be found in He et al. [4]. The 
usage of the folder structure like Sun et al. describe in [5] is not possible for our 
target sites. Modern content management systems prevent from the use of the 
folder structure by cryptic URLs. 
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2 Algorithm 

The structure of a web site is determined by the link structure between its web 
pages. A link in this context is considered to be a link encoded as a HTML tag. 
We do not consider the anchor text of a link as structural information since we 
believe it belongs to the content information. Excluding self-references, we focus 
on hard coded inter-page links represented by a directed cyclic graph. In order 
to create single page descriptions, a crawler extracts key words and follows all 
web-site-internal links in a breadth- first mode. The words are stemmed in order 
to reduce key word diversity by applying the Porter Stemming Algorithm. We 
assume that the most frequent words define the content of the web page. The 
highest ranked words are said to be key words, where their number is propor- 
tional to the text length. Very frequent key words like the company’s name, 
occurring virtually on all pages, are pruned. In [3] we empirically determined 
the usable HTML tags, which we believe to reflect author’s focus. 



2.1 Local Context Page Description 

In order to include the link structure, we combine a page’s set of key words with 
the key words of its direct neighborhood like [1]. The latter is defined as the 
pages, having a direct link to (inedges) or from (outedges) the page in focus. 
We call it the local context (LC) of a page. We perform the combination 
of structure and content by generating a new set of key words. We add the key 
words from the LC to the processed page and accumulate the frequencies over the 
entire key word set including the anchor tags as key words, ordered by frequency. 
The number of words is kept proportional to the text length. We perform this 
process bottom-up with respect to the minimal click path. We interpret the 
results of our algorithm as an intersection between the contents of the LC and 
the particular page. Moreover, we assume that salient words ascend towards the 
root page. 



2.2 A Topic Discovery Algorithm 

By clustering key words we are trying to identify topics. In other words, we 
consider word classes (topics) {zj}f =1 , and model the likelihood of a document 
d (web pages) as follows, 

P(™d) = \\(^2,p{w\z,P)p{z \0d)) nd,w ( 1 ) 

W Z 

where (3 are parameters specifying the probability of words given topics, 9d are 
document-specific parameters that indicate the topics mixture for document d, 
w d is the word list of this document and rid. w is the number of occurrences of 
word w in w d- Then given a corpus of documents d = l,...,n, we have the 
following EM algorithm to estimate the parameters: 
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— E-step: for each document d, we estimate the posterior distribution of topics 
given each word w in it: 



p(z\w,d) 



p(w\z,P)p(z\0d) 

T,zP( w \z,P)p(z\O d ) 



(2) 



— M-step: we maximize the log-likelihood of “complete data” for the whole 
corpus: 

(np( z \ w i d ) i °gn(p( w \ z ’P)p( z \°d)) nd ’ w ^ ( 3 ) 

d Z ' w w ' 

with respect to parameters /3 and {Od\d=n which gives rise to 

Pi,j = p{wi\zj) oc ^2 n d , Wi p(zj\wi, d) (4) 

d 

0j, d = p(zj\d) oc ^2 n d,wP{zj\w, d) (5) 

W 

We perform the E-step and M-step iteratively and at the convergence obtain the 
final estimate of p(w\z) and p(z\d). p(w\z) indicates the probability of occurrence 
of word w given topic z. The algorithm groups semantically related words into 
topics. Intuitively, if several words often occur together (in the same documents), 
then these words are likely to be associated with one topic. p(z\d) indicates the 
probability of topic z for document d. The parameters transform documents, 
which are originally distributed in high-dimensional but low-level word space, 
into low-dimensional and high-level topic space. 



3 Evaluation and Case Study 

We describe measures to evaluate the improvements by the LC. Therefore, we 
evaluate each measurement for both key word descriptions: stand alone and the 
LC. We performed our clustering algorithm on our data with 100 topics (word 
clusters). Where the number was determined by applying SVD on our data. First, 
we provide some basic definitions: The relevant topics for a specific document 
are the topics with the highest probability topn(p(zj\d ), given document di, 
called topic mixture for cl. The number of documents in LC with respect to a 
particular document is \LC(di)\. We are also interested in the number of distinct 
topics which are most relevant for each document in the LC. The number may 
vary from 1 to \LC{di)\, respectively. 

3.1 Evaluation of Topic Clustering 

For our analysis we have used Siemens’ corporate web site: www.siemens.com. 
We crawled 1569 web pages, extracted key word sets (stand alone description) and 
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derived key word set involving the LC. Regarding a particular document, we are 
now able to determine which topic is the most relvant one. Having this topic, we 
then select the documents which are most similar with respect to it. We observed 
in table 1, that the number of shared relevant topics has increased by using the 
LC: instead of one, now 3 of 4 topics are shared. Topic mixture key words: 
For a given document dj, we determine the topic mixture. Each document di is 
represented by a topic vector v,^ = p(z\di) = {zi...z n }. For all elements in the 
vector (zj € Vd) we select the topic key words. They are weighted with their 
corresponding probabilities for that topic. Finally, the probabilities of all topic 
key words are proportionaly accumulated which form the topic mixture key 
words: p(w\z)p{z\di) . This mixture is supposed to describe the document 

which contains one to several topics with different importance. We are now 
interested in the most important key words for this mixture. A word is said to 
be a key word for the topic mixture, if it has a high accumulated value. 



Table 1 . Topic document rate in the LC for our sample. 





stand alone 


LC 


Number of documents of the LC 


21 


21 


Number of distinct topics of the LC 


20 


4 


Number of occurrences of topic 3 in the LC 


- 


11 of 21 


Topic document rate 


0,048 


0,809 



Topic document rate in LC: In order to evaluate the impact of the LC 
based key word sets, we compare the number of distinct topics for the pages in 
the specific LC. As we discussed above, the number of topics in a LC may vary 
from 1 to the number of documents in the local context. So, the topic document 
rate is defined as 1 — _ Consecutively, the more topics are found 

the less is the rate. Note, that in both cases the pages within the LC are the 
same, but in the stand alone description the LC based algorithm from [3] is 
not applied. In table 1 we see that there are almost as many distinct topics as 
documents in the LC of our sample page. Furthermore, the most relevant topic 
of our test page does not occur at all. The number of distinct topics in the 
LC based description (table 1) has decreased significantly, and thus the topic 
document rate for the LC based description is very high. In addition, the most 
relevant topic for the sample page (topic ID 3) occurrs eleven times as most 
relevant topic for pages in the LC. Obviously, our LC based approach reduces 
key word diversity between linked documents. 



3.2 Case Study for LC-Interpretation 

By applying our measures on the data we have shown that the LC based descrip- 
tion improves the result of our clustering algorithm. Now we want to exploit the 
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additional information revealed by the local context. The LC of a selected page 
consists of 21 pages. We compare the most relevant topics for each page of the 
LC - one topic per page. The table shows that from these 21 topics topic no. 3 is 
the most relevant topic. It occurred 11 times representing more than half of the 
pages. Regarding the variety of topics, one sees that the most relevant topics for 
21 pages consist of 4 different topics. This observation is supported by analysis 
of the LC key words. Out of 215 key words one finds 65 distinct key words. 
Concluding these observations, one can characterize this LC as homogeneous. A 
manual inspection of the LC pages affirms this conclusion. 

4 Conclusion 

This paper describes and evaluates the concept of the local context (LC). First, 
we use the LC successfully to smoothen the stand alone key words what we 
have shown empirically. Furthermore, we have shown that by using the LC the 
web site structure gets incorporated into the key word description of a page. 
Consequently, the clustering algorithm benefits from these information which 
identifies topics successfully. In the case study, we have described one applica- 
tion of the results: We are able to characterize a web page and its surrounding 
pages in terms of their content lromogenity. This is only a part of the potential 
applications. Consequently, we will compare structure and content of a web site 
intensively and combine them with an user behaviour analysis. 

Since this approach does not rely on predefined topics, it can be used gener- 
ically. Additionally, we attempt to give more emphasis to the web site structure 
by using an adjacency matrix in the cluster algorithm. 
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Abstract. We examine general and abstract approaches to web engi- 
neering and context-awareness and how they interact with each other. 
This involves considering the appropriateness of approaches to context 
when used by a complex application such as a content management sys- 
tem, while, at the same time, presenting how a content management 
system can use context information to enrich its functionality. We show 
that the integration of such systems is feasible only if, in both fields, 
we make use of approaches based on strong information models. Last 
but not least, we show that the relationship between context engines 
and content management systems is not at all a one-sided client-server 
scenario, but rather a mutually important symbiosis. 



1 Introduction 

As the amount of available digital data continues to grow, vast quantities of in- 
formation surround us in our everyday life. With the abundance of information 
that is accessible to us, the problem no longer is whether information is within 
reach, but rather whether the right information can be delivered in the right 
form to the right person at the right time. Hence, the value of information is 
no longer solely dependent on the quality of the information itself, but is in- 
fluenced by other factors as well. Emerging from classical information systems, 
content management systems have been developed to attain the goal of adequate 
information delivery. These systems extend the traditional functionalities of or- 
ganising and accessing data with means to publish, present and deliver content. 
Even though content management systems were originally developed for the ad- 
ministration of large websites, they have proven to be so successful that a wide 
range of other applications use content delivery technologies nowadays. 

At the same time, a shift in applications to mobile and ubiquitous informa- 
tion environments is clearly noticeable. Applications within this domain require 
highly situation-aware computing and it no longer suffices to simply present the 
information to the user in the right form. Depending on the user’s situation, 
the information must be filtered according to what is important for the user. 
Information is no longer static, but highly dynamic as the situation of the user 
is altering continually according to an ever-changing environment. Context and 
context-aware computing have been recognised as key concepts in reaching this 
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goal and a lot of research has been invested in this topic. Today, application- 
specific solutions and context frameworks exist, but a generic context engine 
with a semantic model for context is still lacking. 

We have developed such a context engine for the management of context 
information that can be coupled with existing applications to augment them with 
the notion of context. We have also developed a content management system that 
supports the delivery of context-dependent content , but does not specify its own 
model for context. In this paper, we present the integration and interplay of these 
two aspects and components and how they complement each other. As we will 
see, the interplay between content and context is not a client-server relationship, 
but rather a symbiosis where both counterparts profit from each other. 

Sect. 2 presents an overview of related work that has been done in the field 
of context-aware computing in general and specifically in the area of content 
management systems. In Sect. 3 we then describe our context engine. Our content 
management system that is used to demonstrate the interplay of content and 
context is outlined at the beginning of Sect. 4 followed with a discussion of how 
context influences the content management system. The opposite direction — how 
the content management system influences the context engine — is presented in 
Sect. 5. Finally, future directions and conclusions are given in Sect. 6. 



2 Related Work 

From the very beginning of the batch processing era, programs have been de- 
signed to produce and present an appropriate output to some given input. Al- 
though nowadays the input could be issued interactively and an application may 
even accept input from arbitrary sources, the principle of the black-box still 
holds. The output is completely determined by the given input. In contrast, 
context-aware applications also use implicit information hidden in the applica- 
tion’s and user’s environment [1], in addition to explicit input, to determine the 
output. 

The influence of user behaviour and, in general, the situation of the en- 
vironment where an interaction is taking place has been identified as context 
and studied by different communities including philosophy, linguistics and social 
sciences. Later people in the field of computer science borrowed the term and 
applied it in various applications [2,3,4]. With the rise of ubiquitous and perva- 
sive computing, context has strongly been associated with information extracted 
from the physical environment. Spatial information such as location, orientation 
and speed or environmental information such as temperature, light and noise 
level are commonly sensed context properties for this application domain. 

Context-aware applications have been designed and implemented such as 
office and meeting tools [5], tourist guides [6,7] and specific fieldwork enhance- 
ments [8]. All those examples propose context models specifically tailored to 
each application domain and directly implemented in the application logic. Many 
common concepts and components had to be implemented in isolation for each 
application. At the same time, some researchers wanted to provide a reusable 
and helpful infrastructure and proposed general context frameworks [9,10]. 
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Based on such a framework, a context-engine can be created to acquire, man- 
age and offer context-relevant information. Applications can then use the engine 
to gather context information which enriches their behaviour. Furthermore, the 
proposals offer an infrastructural solution to the architecture, based on either 
a transparent host/port distributed communication [9] or CORBA [10]. Fur- 
ther, [9] separates the actual sensor from the abstract context which allows the 
decoupling of context acquisition and the application using the context. 

Although the functionality and infrastructure offered by such context frame- 
works simplifies the implementation of context-aware applications, they still lack 
a clear information and conceptual model that describes the context engine. Such 
a metamodel can be used as a reference model for context-engines, increasing 
the understanding of different approaches and enabling a comparison based on 
the concepts introduced in the metamodel. Context information exchange is also 
facilitated by using appropriate mappings to transform context-engine specific 
data to the metamodel. In [11], we present a general information model for con- 
text and, in the next section, we describe a context engine based on that model. 

A general purpose context-engine can be used by any type of application. 
Nowadays, web-based applications represent a major and common class of appli- 
cations as web technologies and infrastructure have become defacto standards for 
many application environments. In the field of web engineering, the first steps to- 
wards context-awareness were made by introducing adaptation concepts directly 
into hypertext models [12,13]. Such Adaptive Hypermedia Systems (AHS) are 
based on various user characteristics presented in a user model. Most of the 
early examples focus on information collected from the user’s click stream. In 
[14], user data, usage data and environment data is distinguished. An exten- 
sive discussion on the field of AHS and related work can be found in [15]. We 
should mention here that user characteristics such as presented in AHS are not 
always necessarily describing context, but rather the user profile in general. For 
example, user preferences such as colours, styles and interests might be part of 
the current context, i.e. they contribute to the dynamic characterisation of an 
interaction situation, but this does not have to be the case. 

AHS employs a user model to capture the adaptive relevant properties. After 
each interaction, the user model is updated and, based on adaptation rules, 
appropriate output is generated. Adaptation is applied to the basic concepts of 
hypermedia models, namely, component and link. Rules are defined to control 
the composition of components (fragments) or even alter link presentation. 

Due to the fact that components encapsulate both notions of content and 
presentation in an indistinguishable manner, it is not possible to adapt only 
one of them in isolation. This is unfortunate if one considers that a major 
adaptation requirement of web applications is context-dependent presentations. 
When evaluating AHS, we are confronted with the situation found in other 
application-specific approaches. The user model is inspired and specified with 
the browser/server environment in mind. Thus, no general approach to context 
is taken with important properties such as context history, sensor generality, 
quality etc. 
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To provide a better solution, model-based approaches to web site develop- 
ment have recently been studied with respect to adaptation [16] and context- 
awareness [17,18,19]. Based on strong and abstract information models, these 
approaches have the potential to exploit context-awareness in many dimensions, 
keeping the solutions as simple as possible. In [16], delivery channel charac- 
teristics can be used to influence the hypertext and navigational model of the 
original WebML model [20]. In this approach, no explicit context model is used. 
Instead, general data modelling techniques are made available to manage con- 
text information. Although the presentation cannot be adapted explicitly, it can 
be influenced by adapting similar content units bound to different presentation 
templates. 

In OMSwe [17], the database management system OMS Pro [21] is extended 
with versioning and basic content and presentation management facilities to 
empower a web development environment. All relevant information, from the 
development process to implementation and application data, is managed by 
the database system. Context information is provided in the form of characteris- 
tic/value pairs from a context gateway component. This information builds the 
request state, based on which, the database will retrieve the most appropriate 
versions of the objects involved in the response, extensible Content Management 
(XCM) [22] takes the approach one step further and defines a full-strength con- 
tent management system with well defined general information concepts. Based 
on this approach, we present here the interplay between this system and the 
general context engine that we developed. Details of XCM are given in Sect. 4. 

3 Context Engine 

In [11], we consolidated research in the field of context and context-awareness and 
presented a model for a generic context engine. The basic and abstract concept 
of context is influenced both by the application and framework requirements, 
as well as the problems presented by Dourish in [23]. In this section, we use an 
example to describe how our context engine works and how application-specific 
models can be specified. 

Each entity of an application can potentially have a context. An application 
schema, for instance, could define the concept of a room. The context of a room 
could be described by its physical properties such as the temperature and the 
light intensity. For each point in time, the room’s context is composed of the set 
of values over its context properties: e.g. temperature=30 and light=40. 

Obviously, the values 30 and 40 do not provide any information about the 
scales that are used. Moreover, a context engine supplies context information to 
many different applications, thus making a quantification even more important. 
We therefore use a type system to define concepts within the context engine. 

The system distinguishes four kind of types. Base types define common values 
such as string, integer, boolean etc. The model allows base type restrictions 
as used in XML Schema. Hence, we can create base type restrictions with new 
names that represent values with specific semantics. In the example below, we 
define a base type Celsius as a restriction of real. Restrictions of a type are 
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expressed through a constraint that checks the validity of the values acceptable 
for the defined type. 



btype Celsius is real { 

constr: ’ self(S), S > -273.16’; 

}; 

Our prototype system is implemented in Prolog and the context definition 
language (CDL) used here has been developed to define concepts and their types 
for an application-specific context schema. Operational components of CDL are 
defined in Prolog. For the above example, the constraint is a predicate that 
evaluates to true if the given value is legal. To check if a value is appropriate as 
a Celsius temperature, the value is first retrieved using the predicate self/1 
and then checked to be greater than —273.16. 

Apart from the base types, the context model uses references to application- 
specific types to define and control application entities. Apart from a uniquely 
identifiable application type, the context engine does not deal with, and has no 
control over, the values composing an application entity. Thus an application ref- 
erence type is sufficiently defined by the pattern applicationID : typelD. The 
third class of types are the bulk types which designate sets of values of a given 
member type. Finally, the model supports context types which define the com- 
position of the context itself rather than its values. As a common composite 
type, context types are defined over a set of attributes of a given type. In the 
following example, we define a context type ctxvtemperature with one attribute 
temperature of type Celsius. 



contextType ctx_temperature characterises app:physical_obj { 
temperature: Celsius; 

}; 

Optionally, the context type designates the type of entity to which it can 
be bound. In the above example, the defined context poses the constraint to 
characterise application entities of the type physical_obj defined in applica- 
tion app. Application types could be defined in an is-a hierarchy, designating a 
compatibility among them (not shown in this example). 

After declaring the types, an application can create context elements and 
bind them to some application entity. In the following example we create the 
temperature context for app : roomA32, which is an instance of app : room, subtype 
of app : physical_obj . 



context c_roomA32_temp: ctx_temperature describes app:roomA32 



Context elements are either queried directly from client applications or re- 
ceived through an event notification mechanism. Such clients play the role of 
consumer with respect to the context engine. On the other hand, a sensor ab- 
straction exists that encapsulates the context acquisition mechanism. Sensors are 
well defined components that are initialised with some parameters and bound 
to context elements. Sensors could either be hardware or software in nature. A 
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hardware sensor could be a thermometer and a software sensor could be a pro- 
gram that extracts the current application status with respect to an interaction. 

To allow reusability and separation of concerns, we introduced the concept 
of a sensor driver. It holds the acquisition logic of the sensor. Multiple sensors 
can be instantiated based on a single sensor driver. The next example defines a 
temperature sensor that is connected to the serial port of a computer and pro- 
vides temperature context information based on the ctx_temperature context 
type. The last statement initialises the actual thermometer based on the given 
sensor driver, providing context information to the c_roomA32_temp context. 



sensorDriver tempSensor (serial_port : integer): ctx_temperature ; 
sensor s .thermo 1: tempSensor (1) provides c_roomA32_temp; 



In addition to the sensor driver definition, an implementation is necessary. 
For our prototype, it is coded in SICStus Prolog [24] and bindings to Java are 
used in cases where the actual sensor offers a Java API. Examples of such sensors 
are software sensors that use information collected from the content management 
system. More details on such sensors will be given in Sect. 4. For a context-aware 
news application, we have developed a prototype sensor driver for the recognition 
of people in a room. It is based on the ARToolKit [25] augmented reality library 
and recognises people through special tags that are recognised by the ARToolKit 
in a stream from a small video camera. 

Having presented our context engine, we now go on to explain in the next 
section how it can be used to make an existing application context-aware. 



4 From Context to Content 

To demonstrate the interworking of the previously described context engine with 
an application system, we show in this section how it can be integrated with a 
content management system. Further, we outline how information flows from the 
context engine to the content management system and vice versa. As a content 
management system, we have chosen our own extensible Content Management 
(XCM) [22,18]. The current version of XCM is implemented in Java and based 
on OMS Java [26], an object-oriented database management system based on 
the Object Model (OM) [27], a model that integrates object-oriented and entity- 
relationship concepts. The original version of XCM supports only a rather simple 
implementation for context. We are currently working on the extension of XCM 
for support of the context engine that we have described in the previous section. 

XCM has been designed as a platform that provides support to applications 
requiring features typical of content management such as user management, 
workflows, personalisation, multi-channel delivery and multi-variant objects. At 
the heart of the system stands the separation of content, structure, view and 
layout. The concept of content allows data to be accumulated into content ob- 
jects and stores metadata about this content. As it is often required that the 
same content object may be delivered in different variations, the system sup- 
ports what we call “multi- variant objects”. For example, a news article may 
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have multiple variants that correspond to the different languages in which it 
is available. Multi-variant objects however can comprise much more than the 
simple dimension of language. Other dimensions are often required, such as the 
target group of users or whether it is free or premium content for which users 
have to pay. Our system does not predefine the possible dimensions, but rather 
provides support to handle any annotation of content variants that makes sense 
in a given application domain. To achieve this, the concept of a content object is 
separated from its actual representation while still allowing for strict typing. In 
Fig. 1, a metamodel is displayed that shows how XCM represents multi-variant 
objects. The notation and semantics is that of the previously mentioned Object 
Model (OM). 




Fig. 1 . Metamodel for XCM Multi- Variant Objects 

At the top of the figure, the separation of the concept of an object and its 
actual content is clearly visible in the form of Objects linked to a non-empty set 
of Variants by means of the association hasVariant. Each variant of a multi- 
variant content object can be described by Characteristics that are linked to 
the variant over the association describedBy. A characteristic is simply repre- 
sented as a (name , value) tuple. For example, to annotate a variant for english, 
one would simply associate it with the tuple (language. name, english). As 
we will see later, characteristics play an important role in context-aware con- 
tent management applications. XCM also manages metadata about the types 
of content objects and their variants. Thus, in the metamodel, objects as well 
as variants are associated with Types using the hasType and variantType as- 
sociations, respectively. This information can then be used by the system to 
determine whether the type of a variant matches the type of the corresponding 
object. As the cardinality constraints indicate, an object can have more than 
one type to support polymorphism and multiple instantiation. 

XCM uses the concept of structure to build content hierarchies from multi- 
variant content objects. In a content management system, structures are required 
to build complex objects such as pages, collection of pages or “folders”. Our 
system uses the very simple component-container approach to build these struc- 
tures. As a container can hold a number of components and other containers 
as well, arbitrary tree-based structures can be represented. Structure objects, 
i.e. containers, form the inner nodes of the tree, whereas the actual content is 
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located in the leaves. Separating the structure from the content makes possible 
different access patterns to the same content. 

Personalisation in XCM is achieved through the concept of a view. A view 
decides which aspects of a multi- variant content object are presented to the user. 
Further, it can aggregate other information to the object based on the schema 
of the content, thereby augmenting it. Hence, our view concept is not unlike 
that found in relational database systems. Managing different views for different 
users or situations effectively provides support for personalisation and custom 
content delivery. 

Finally, the concept of layout encapsulates the graphical representation of 
the content. The layout is applied to container and content objects by means 
of templates that match the type of the target object. As for all four basic 
concepts of our system, layout objects can have multiple variants for different 
requirements. For example, a template to render a news article can have one 
variant to produce HTML and another variant to display WML. Hence, layout 
objects are required to support multiple presentation channels. 

Based on this overview of the basic elements of XCM, we now go on to 
describe the integration of the context engine into the system. As mentioned 
before, our context model allows application entities to be linked to context 
elements within the context engine. In the example of the content management 
application, it is intuitive to link the application concept of Sessions to various 
context elements such as the user’s identity and language, the version of the 
browser and other information about the current situation of the requestor. As 
sessions are already used in content management systems to store values essential 
to the current user, it is only natural to use this concept and augment it with 
context information. 




Fig. 2. Conceptual Model for the Context Binding 

Figure 2 shows the model of the binding between XCM and the context 
engine. The parts that physically belong to the context model of our context 
component are represented with dashed lines. The four basic concepts have been 
unified with a common super concept XCMElements which depends on the values 
of a given session. The separation between an object and its variants has been 
collapsed into a box with multiple shades to indicate that these objects can have 
multiple variants. From the model, it is apparent that not only content, but 
also structure, view and layout can have multiple variants and thus adapt to 
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context. Again, each variant can be described with characteristics as shown on 
the left-hand side of the figure. 

This conceptual link between context elements and the application concept 
of a session is materialised using the context type definition given in the example 
below. The displayed Prolog code first creates a base type for physical internet 
addresses by means of a restriction on the type string. Then a context type 
is created to represent the browser that is used in a given context. The actual 
binding is defined by the third declaration. It defines a type ctx_session which 
is linked to the concept xcm: session within XCM. It comprises four attributes 
that represent the context information available from the sensors. The attribute 
user provides the identity of the user of the current session, lang its language 
and browser the name and version of the browser used. Finally, the attribute 
host gives information about the physical address from which the content man- 
agement system is accessed. 

btype ip is string { 

constr: ’ self(S), split (S ,L) , \+ (member (X,L) , \+ conv_integer(X,_)) ’ ; 

}; 

contextType ctx_browser { 
name: string; 
version: string; 

}; 

contextType ctx.session characterises xcm: session { 
user: xcm: user; 
lang: language; 
browser: ctx_browser; 
host: ip; 

}; 

Having explained how the context engine is bound to the content manage- 
ment application, we now go on to discuss how such a system reacts to the 
information supplied by the context component. In XCM, context information 
is used to select the appropriate variant of an object, i.e. the context information 
is compared to the characteristics of each variant of such an object and the best 
matching version is selected. In practice, this can prove to be quite difficult, as 
the characteristics available to the system need not fully match the incoming 
context information. It is often the case that the context information comes at 
a higher level of detail than the metadata stored in the content management 
system. Of course, the opposite case that the metadata is more precise is also 
possible. In these cases of “under-” and “over-specified” context information, the 
matching process uses heuristics to select from the set of possible variants or to 
select a default variant if no match is found at all. A similar matching process 
together with heuristics is used in [17,28]. 

For the mapping of context information to characteristics to work, the names 
of the (name, value) tuples have to match. This means that some sort of con- 
vention, taxonomy or ontology has to be used that assures a uniform naming 
scheme. As the definition of such conventions is clearly beyond the scope of 
this work, we adopt the simple approach of what we call “property paths” . A 
property path locates a value inside a context element and can be derived from 
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its context type definition, e.g. the value of the browser version in the current 
session would be identified by the path ctx_session. browser . version. 

Further complexity arises from the fact that XCM supports not only sim- 
ple values for characteristics, but also sets and ranges. This is used to spec- 
ify, for instance, a set of matching browser versions with the characteristic 
(ctx_session. browser .version, [5.0, 5.5, 6.0, 6.1]) or a time period 
when a variant of an object is valid with (valid, [1/1/2004. .31/1/2004]). 
XCM also provides the notion of mandatory and “show-stopper” characteristics. 
In contrast to unconstrained characteristics, these categories of properties are 
not freely matched to the context information, but instead are treated specially. 
A mandatory characteristic takes precedence over all other non-mandatory val- 
ues, whereas a “show-stopper” property would not allow a certain variant to be 
selected if the context value does not match that of the characteristic. 
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Fig. 3. Overview of the Architecture 

Having discussed the integration of the context component into our content 
management system, we can finally give an architectural overview of the whole 
system. Figure 3 shows a graphical representation of all components. At the bot- 
tom, the user is shown that communicates with the content management system 
through a browser requesting pages. The content management system then in- 
terworks with a private context component to manage the context proprietary 
to the system. Further, the private context component is connected to a shared 
context component that manages context for several applications and provides 
a global notion of context. This shared context component is connected to the 
sensors as shown at the top of the figure. The two context components and the 
content management application are conceptually connected by the model that 
is common to all three of them. As the figure also indicates, information can 
always flow in both directions. We discuss the opposite flow of information in 
the next section. 
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5 From Content to Context 

In the previous section, we described how the content management system can 
use the context engine to provide context-aware content for a user. In this section, 
we describe the reverse relationship. The content management system can also 
act as a provider of information for the context engine. This context information 
can then be used either by the content management system itself, or by another 
context-aware application of an information environment. 

In Sect. 3, we introduced the abstract concept of a sensor. A sensor is basi- 
cally any piece of software or hardware that generates context values for a given 
context element. Sensors can be classified into two categories, depending on the 
environment that they target: Physical sensors acquire context information from 
the physical environment, e.g. from a room, the human body or a physical ob- 
ject. Software sensors, on the other hand, gather information from the virtual 
environment formed by the logic and data of an application. The content man- 
agement system is such an application to which software sensors can be applied. 
As XCM is implemented in Java, the corresponding software sensors would be 
implemented in the same language. The binding of the sensors to the context 
engine is done through the mechanism of a sensor driver as mentioned in Sect. 3. 

The concepts of interest are the same for both physical and software sensors. 
From the main areas described in [29], the user environment is of special im- 
portance for information environments. Whereas the physical part of the user 
environment (e.g. location and orientation of a person) is usually relatively easy 
to acquire with appropriate sensors, the mental state of a user is much harder 
to obtain. The content management system is able to provide very valuable con- 
text information in this area such as what the user is currently working on, 
which topics are of interest, how busy the user is and whether they are currently 
working on a precise topic or just browsing an information space. 

Context information can be extracted from application data, content data, 
the local context model of the content management system or the user sessions. 
Some of this context information, e.g. the level and type of activity, is indepen- 
dent of the actual application data in the content management system and relies 
only on its metadata. Software sensors can be provided that extract this con- 
text information in a generic way. However, for the other examples mentioned 
above, the actual data model of the application is very important. The fact that 
we have an expressive semantic data model in our content management system 
greatly simplifies the task. Actually, the extraction of semantic context infor- 
mation would be very difficult, if not impossible, without a proper application 
model in the content management system. Imagine, for instance, a news applica- 
tion where individual news messages are categorised with respect to their topics. 
In some traditional content management systems that do not support customised 
data models, this could be modelled with a page for each topic containing a list 
of news messages in the form of paragraphs, links etc. and a page for each news 
message with more details. In this setup, it is not possible to extract the topics 
in which the user is currently interested. As we have described in Sect. 3, in 
our content management system, the application data is not stored in the form 
of paragraphs, links and pages, but rather according to a semantic application 
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model. A customised software sensor could thus easily retrieve the categories of 
the news messages that the user is currently browsing and store corresponding 
entries in the context engine. In contrast to other context frameworks such as [9], 
as described in Sect. 3, we are also able to use application objects as context 
values through reference types. This means that the content management system 
can store the actual object that represents the category in the context engine, 
rather than just the name of the category. 

It is important not to confuse the extraction of context information with 
general data mining, e.g. for user profiling. Similar technologies and algorithms 
can be used for both of them, but whereas data mining focusses on general facts 
and information, for example about users, context mining is ouly concerned 
with the current state of the user. This might include historical information, but 
usually, context information is of interest for a limited time only. 

The extracted context information can be used for a variety of applications. 
Of special interest to us are information environments and corresponding plat- 
forms that integrate multiple context-aware applications for a physical location 
or virtual community. We have implemented a context-aware news application 
for such an information environment that uses a content management system 
for the delivery of information and adapts according to the people in a room. 
Currently, we are working on the extension of this news application towards the 
inclusion of virtual rooms and the integration of the context engine described 
previously. 

Another example are community awareness applications that visualise the 
presence and activities of other users. This is context information of the user. A 
content management system as part of an information environment of a commu- 
nity could provide some of this context information, for example, for an applica- 
tion such as the coutext-aware instant messaging and chat application described 
in [30]. It visualises a list of users that are present along with their current 
state (present, busy, free-for-chat etc.). Another interesting example is the area 
of Computer-Supported Cooperative Work (CSCW) or collaborative systems in 
general. In this context, the content management system could also provide con- 
text information for a community of users, such as users working on specific 
topics, rather than just for individuals. This information could be displayed by 
other coutext-aware applications or the context management system itself. 

6 Conclusions and Future Work 

We have discussed the interplay between context and content in terms of the 
relationship between the basic concepts of context engines and content man- 
agement systems, respectively. We used the example of our own content man- 
agement system to demonstrate how our generic context engine enabled us to 
seamlessly adapt all dimensions of content delivery — content, view, structure and 
presentation — through a process of matching context to multi- variant objects for 
these dimensions. 

Moreover, we showed that a content management system can be, not only a 
client to the context engine, but also a potential provider of context information. 
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Our system XCM exhibits two features that make it an ideal component for de- 
termining the computation context: It consolidates well organised information 
from arbitrary sources and acts as a bridge between the user and the organi- 
sation through multiple communication channels. We showed how this crucial 
and complete information provides context and how we use XCM as a software 
sensor to the context engine. 

We are currently working on a platform for a context-aware information 
environment (SOPHIE) that combines features from pervasive computing and 
information systems. The SOPHIE platform is based on content management 
technologies and context engines and represents an important step forward in the 
integration and extension of these technologies to cater for information delivery 
and interaction in environments that span the physical and digital worlds and 
involve multiple interrelated users and devices. 
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Abstract. In this paper, we integrate WebML, a high-level model and technology 
for building server-side Web applications, with UML-Guide, a UML-based 
system that generates client-side guides for the adaptation of Web applications. 
The combination of the two systems is shown at work on an e-learning scenario: 
WebML is the basis of the specification of a generic e-learning system, collecting 
a large number of learning objects, while UML-Guide is used for building 
company-specific e-learning curricula. The resulting system can be considered 
an “adaptive hypermedia generator” in full strength, whose potential expressive 
power goes beyond the experiments reported in this paper. 

Keywords: Personalization, UML, WebML Modeling, Web Engineering. 



1 Introduction 

In recent years, the control of Web applications has moved from the client to the server 
side, leading to more economical, structured, and well engineered solutions. In particular, 
the model-driven approach, as advocated in [3,6,11,17] has proved very effective in 
extending the classical methods and best practices of Software Engineering to the Web. 
Design methods now concentrate on content, navigation, and presentation design, which 
are orthogonally developed by means of specialized abstractions and techniques. 

While server-side solutions are dominant, yet bringing some intelligence to the client 
may be highly beneficial in some cases [16,18]. Client-side solutions can reveal as being 
more dynamic, more adaptive, and protective for sensitive user data. They may be very 
effective for “remembering" the local context or being aware of the local peculiarities 
of the interaction. Also, a clear separation of concerns between the client and the server 
may lead to interesting business opportunities and models. 

This paper explores the combination of two existing approaches to the engineering 
of Web applications. We use the WebML method [3] and its development support en- 
vironment [4] for generating the application server-side “backbone”. We then integrate 
such a backbone with UML-Guide [9], a client-side personalization engine that dynam- 
ically generates additional interfaces and user guides for personalizing the application’s 
fruition, by managing user profiles and context-sensitive data at client side. 



N. Koch, P. Fraternali, and M. Wirsing (Eds.): ICWE 2004, LNCS 3140, pp. 201-214, 2004. 
(c) Springer- Verlag Berlin Heidelberg 2004 
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The proposed approach capitalizes on the use of two systems that both start from high- 
level abstractions, and are both capable of automatic deployment of the implementations: 

- The WebML method is based on the use of high-level concepts, such as the notions 
of entity and relationship to denote content, and of page, unit, and link to denote 
hypertexts. These abstractions are automatically turned into implementation artifacts 
by means of WebRatio, a tool for the automatic deployment of Web applications [3], 

- UML-Guide is based on the use of UML state diagrams, whose nodes and arcs — 
representing states and transitions — are turned into XMI specifications. A client-side 
translator, written in XSL, turns such specifications into a user interface facilitating 
the adaptive use of the application [9]. 

Coupling WebML and UML-Guide yields the following advantages: 

- The use of high-level WebML abstractions in the context of UML-Guide enables 
the specification of a powerful client-side personalization engine. The resulting 
application generator can be considered an “adaptive hypermedia generator” in 
full strength, whose potential expressive power goes well beyond the experiment 
reported in this paper. 

- The tools prove to be highly complementary and easily integrated, as it is sufficient 
to reuse concepts of WebML inside UML-Guide to provide concept interoperability, 
and the URL generation technique of the WebML runtime inside the UML-Guide 
XSL code to provide systems interoperability. 

- The use of UML-driven methods in conjunction with WebML is by itself a very 
interesting direction of research, aiming at the integration of UML, the most con- 
solidated software engineering method (and related technology), with WebML as a 
representative case of new, hypertext- specific models and techniques. 

1.1 Driving Scenario 

In order to exemplify the integration of the two methods, we refer to an e-learning 
scenario, in which a courseware company develops and distributes a vertical applica- 
tion for e-learning, running on the company’s server, specified and developed through 
WebML 1 . The vertical incorporates learning objects in the format of lessons, exercises, 
tests, definitions and examples for computer science, arranged according to the ACM 
categories 2 , and learning paths with checkpoints for the learner. Thus, such a vertical has 
learning objects as content, and navigation mechanisms, such as guided tours or indexed 
accesses to pages based on broad categories, enabling a generic user to access such a 
content though predefined navigation paths. 

The vertical is used by Small-Medium Enterprises (SMEs) wishing to build person- 
alized e-learning curricula, to be used by their employees for focused training activities. 
We assume that each SME has a clear instruction goal (for example, teaching its employ- 
ees how to integrate Java programming into Oracle 9i), and that it can use UML-Guide 

1 This scenario is suggested by the ProLearn Network of Excellence, whose main focus is the 
enhancement of professional e-learning methods and technology; see 

http : / /www . prolearn-pro j ect . org. 

2 See http : //www. acm. org/ class/1998/ 
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to specify it in UML; we assume that UML state diagrams, together with a vocabulary 
listing all the learning objects available in the vertical, may be an easy-to-use interface 
for the SME designer. UML-Guide specifications select the concepts to be covered in the 
learning paths, as well as the workflow driving the student in the learning process. We 
also assume that each SME has a clear view of its employees’ competencies, and thus 
is able to constrain possibilities in the learning paths by adaptation rules based on such 
competencies. These rules enable adaptive content selection from the WebML vertical 
and also enable to adaptively indicate, show, and hide links in the learning path, and 
adaptively customize their targets. 

1.2 Paper Organization 

The paper is organized as follows. Section 2 introduces the WebML component, by 
providing an overview of the WebML method and the WebML-based specification of 
the vertical e-learning application. Section 3 introduces the UML-Guide method, and 
the specification of the client-side personalization for the vertical e-learning. Section 4 
illustrates the integration of the two methods by means of an architecture where the appli- 
cation server-side code is generated through WebML, while personalization with respect 
to specific learning goals is managed at client-side through UML-Guide. Section 5 then 
describes the user interface generated for the integrated application. Sections 6 and 7 
illustrate some related work and draw our conclusions and future work. 

2 WebML Component 

WebML is a visual language for specifying the content structure of a Web application and 
the organization and presentation of contents in one or more hypertexts [3], The design 
process based on WebML starts with the specification of a data schema, expressing the 
organization of contents by means of the Entity-Relationship primitives. The WebML 
Hypertext Model allows then describing how contents, previously specified in the data 
schema, are published into the application hypertext. 

The overall structure of the hypertext is defined in terms of site views, areas, pages 
and content units. A site view is a hypertext, designed to address a specific set of re- 
quirements. It is composed of areas, which are the main sections of the hypertext, and 
comprise recursively other sub-areas or pages. Pages are the actual containers of infor- 
mation delivered to the user; they are made of content units that are elementary pieces of 
information, extracted from the data sources by means of queries and published within 
pages. In particular, as described in Table 1, content units denote alternative ways for 
displaying one or more entity instances. 

Their specification requires the definition of a source (the name of the entity from 
which the unit’s content is extracted) and a selector (a condition, used for retrieving the 
actual objects of the source entity that contribute to the unit’s content). 

Within site views, links interconnect content units and pages in a variety of config- 
urations yielding to composite navigation mechanisms. Besides representing user nav- 
igation, links between units also specify the transportation of some information (called 
context) that the destination unit uses for selecting the data instances to be displayed. 
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Table 1 . Some basic WebML content units. The whole set of units is described in [3]. 



Unit name 



Visual Notation 



Description 



Data unit 



0 

6 



It displays a set of attributes for a single entity in- 
stance. 



[Selector] 



Multidata unit 



©’© 
© E3 

“ 3 “ 



It displays a set of instances for a given entity. 



[Selector] 



Index unit 



It displays a list of properties, also called descriptive 
keys , of a given set of entity instances. 



[Selector] 



Hierarchical Index unit 



Scroller unit 



Hierarchicallndex 



5 

[Selector!] 
NEST Entity2 
[Selector] 






A variant of the index unit, which displays list of 
properties of instances selected from multiple enti- 
ties, nested in multi-level three. 



It represents a scrolling mechanism, 
based on a block factor, for the elements 
in a set of instances. 



[Selector] 



WebML-based development is supported by a CASE tool [4], which offers a vi- 
sual environment for drawing the WebML conceptual schemas, and then supports the 
automatic generation of server-side code. The generated applications run in a standard 
runtime framework on top of Java 2 application servers, and have a flexible, service-based 
architecture allowing components customization. 

2.1 WebML Specification for the Vertical E-learning 

The data schema of the vertical e-learning application is centered on the concept of 
learning object. As reported in Figure 1, the LQ entity represents descriptions of learning 
objects, by means of attributes inspired by the LOM standard 3 . Among them, the attribute 
type expresses the different types of LOs (e.g., lectures, lecture modules, definitions, 
exercises, tests) published by the vertical application. Each LO has associations with 
other LOs : for example, a lecture module can be associated with some related definitions, 
exercises, examples, or tests. The entity Content then represents the contents (texts, 
images, files) LOs consists of. In order to facilitate LO access, the schema also includes 
the entity Category: it stores the ACM categories that classify the LOs published by 
the e-learning application. 

3 http : / /ltsc . ieee . org/ 
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Fig. 1. WebML Data schema for the vertical e-learning application. 




Fig. 2. The WebML specification of the hypertext interface for the vertical e-learning application. 



Figure 2 reports a simplified excerpt of the WebML hypertext schema defined for 
the vertical e-learning application; it refers to pages for selecting a lecture module, and 
accessing its contents as well as associated definitions, exercises examples, and tests. 
The lecture module selection is operated by means of a navigation chain, in which 
users progressively select a subject category (Categories page), then a course that 
refers to the selected category (Courses page), then a lecture (CourseLectures page), 
and finally the lecture module they are interested in (LectureModules page). Pages 
Categories and LectureModules are marked with an “L" label, which indicates that 
they are landmark pages. This property represents that the two pages will be reachable 
from any other page of the hypertext, by means of landmark links. 

Contents of the selected lecture module are shown in page LectureContent. As rep- 
resented by the Module Scroller unit, users can browse lecture modules in a Guided 
Tour navigation that allows moving forward and backward in the (ordered) set of mod- 
ules available for the currently selected lecture. For each module, the data unit Module 
Title shows the title and a short description of the learning object, the Contents mul- 
tidata unit shows texts and images that compose the module, while the Definitions 
hierarchical index shows titles of the definitions associated with the module and, nested, 
the corresponding contents. Three index units then show the lists of examples, tests and 
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exercises available for the current lecture module. The selection of one item from such 
lists leads users in a different page where the corresponding contents are displayed. 

The presentation of page LectureContent, as generated by the WebML code gen- 
erator, can be seen in the right frame of the Web page depicted in Figure 7. 



3 UML-Guide Component 

3.1 UML-Guide Overview 

UML State diagrams [12] are used in UML-Guide for modelling the user navigation in 
a hypertext. Each state represents the production of a given information chunk on the 
device observed by a user, and each state transition represents an event caused by user 
interaction that leads to the production of a new chunk of information. State diagrams 
therefore provide an abstraction of hypertext trails, where each trail can be adapted by 
taking into account the user background, level of knowledge, preferences and so on [9]. 
In this way, UML state diagrams are a suitable interface for UML-Guide, whose primary 
purpose is to build adaptive hypermedia systems. 

Atomic states can be grouped into superstates. States usually refer to concepts of an 
application domain; thus, they can correspond to the representation of WebML pages or 
page units, which enable the viewing of the information entities within the WebML data 
schema. 

Parallel substates represent information chunks to be presented simultaneously. Fork 
and join pseudostates are used respectively for splitting and joining computations and 
enabling parallelism. The SyncState pseudostate is used for synchronizing substates of 
parallel regions. 

Transitions represent active interconnections between information chunks, and usu- 
ally correspond to associations in the application domain model (thus, they can corre- 
spond to WebML links, that interconnect pages and units, and in turn depend upon the 
relationships of the WebML data model). Events raise transitions in a state machine; they 
include user-generated or system-generated events, and the latter include time events. 
Guards can be used to constrain transitions by adaptation rules. Usually, they consist of 
a predicate over user profile attributes or context information. 

Actions can be performed after a transition is raised and before entering a state. Also, 
transitions can be associated with side effect actions, whose effect is, for example, the 
modification of a user profile, or the choice of presentation styles for a given chunk 
of information. Actions can also process parameters used in guards of outgoing part of 
branches. Side effect actions, as well as adaptation rules, can be assigned to entry, exit, 
and do actions of states. 

Tagged values are domain- specific properties used to extend the semantics of ele- 
ments in UML diagrams. These values can refer, for example, to concepts of the structural 
model of the application domain, or to specific terminologies which might be useful to 
identify additional navigation requirements. We will make extensive use of tagged values 
for linking UML diagrams of UML-Guide to WebML concepts, as illustrated in Sec- 
tion 4. 
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Fig. 3. A navigation model for a Java tutorial in the UML state diagram notation. 



3.2 UML-Guide State Diagram for E-learning 

The UML-Guide state diagram of Figure 3 illustrates a personalized learning environ- 
ment for teaching object-oriented programming in JAVA, borrowed from a well-known 
Sun tutorial 4 . The chosen personalization example focuses on link adaptation; other 
adaptation aspects are covered in [9]. 

The tutorial starts with an overview of available lectures, as represented by the 
Overview state, which summarizes the available lectures in the tutorial, as specified by 
the Summary value in the LearningPresentation tagged value. It also presents the 
high level tutorial steps (Tutorial value in the CourseStructure tagged value). Links 
from the overview point not only to the first section of the tutorial, but also to the other 
main sections; all these links, except the first one, are associated with guard conditions 
that check that the user has enough knowledge to jump directly to the respective lectures. 

The next step from the Overview is a lecture on the Object Oriented Pro- 
gramming Concepts. This state is accessible without any prerequisite on background 
knowledge; it is a composite state, containing five steps, represented by four sub- 
states: What is an Object, What is a Message, What is a Class, Relations 
to Code, and Questions. The Relations to Code state also shows an entry 
procedure addressing content level adaptation. The procedure applies to a learning 
step about building programs; it states that if the current user does not have sufficient 

4 See http : //java. sun. com/docs/books/tutorial/ java/ index.html. 
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+Name : String 
|+ld : String 



+SetLOK(in competence, in LOK, in learningExperience) 
|+Currentl_OK(in competence) : double 



Performance 



+Competence : String 
+LearningExperience : String 
+RecordedDate : Date 
-PerformanceValue : double 
+PerformanceMetric : String 



Fig. 4. A user model for the Java tutorial. 



knowledge on basic concepts about object-oriented programming procedures, then learn- 
ing content on procedures will be added. 

The next step from the Object Oriented Programming Concepts is the com- 
posite state Language Basics. The transition between the two states features a next 
event and a guard. The guard specifies a link level adaptation rule, saying that the link 
is recommended when current user level of knowledge is greater then zero. The other 
learning steps modelled in the state diagram can be interpreted similarly. 

The personalization specification within state diagrams is based on the user model 
depicted in Figure 4. It is inspired by the LTSC IEEE 1484.2 Learner Model WG Standard 
proposal for public and private information (PAPI) for learner 5 6 . The user model is com- 
posed of the classes User and Perf ormance, plus an association expressing that a learner 
can have several performance records based on the acquired LearningExperience and 
Competence. 

The Performance class stores the user’s level of knowledge about the concepts 
described by the tutorial. This value is the one used for determining if a transition into 
a new state is appropriate and must be suggested to a given user. For example, the 
following condition: 

[CurrentUser.CurrentLOK(‘ ‘Classes and Inheritance’ ’)>0] 

is a guard that in the state diagram determines wether a link can be followed between the 
Classes and Inheritance state and the Interfaces and Packages state, based 
on current user level of knowledge. The Perf ormance class maintains as well the value 
of competence, recorded date, and metrics used to measure level of competence. 

The User class provides operations to set and get the acquired level of knowledge 
or level of competence. These operations are used in guards and actions for adaptivity 
rules, and for updating learner profile. For example, in the state diagram of Figure 3, 
the user level of knowledge about “Classes and Inheritance" can be acquired either 
in the Object Oriented Programming Concepts lecture or in the Classes and 
Inheritance lecture. Exit procedures of these states indeed contain similar update 
operations, as the one which follows: 

CurrentUser . SetL0K( ‘ ‘ Classes and Inheritance ’ 1 , 0 . 2 , Content) . 



5 http : / /It sc . ieee . org/ archive /harvested- 2003- 10/working_groups/wg2 . zip 

6 For a more detailed learner profile, used e.g. in EU/IST Elena 

(http : / /www . elena-pro j ect . org). the reader is referred to the learner RDF bindings web 
site at http : //www. learninglab . de/~dolog/learnerrdf bindings/. 
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In UML-Guide, state diagrams are used as input for visualizing navigation maps, 
whose structure is made of documents (nodes), composite nodes (folders), links (ar- 
rows), and parallel regions (dashed boxes). State diagrams are edited by means of the 
commercial tool Poseidon 7 ). The navigation map is then generated through a transfor- 
mation method [9], whose input is the state diagram encoded in XMI (as produced by 
Poseidon), and whose output is the map. 

4 Integration of WebML and UML-Guide 

The integration of WebML with UML-Guide proposed in this paper aims at composing 
a generic “vertical e-learning” WebML application with a UML-Guide that is focused 
on a specific learning goal. We offer to the users of the composite system the standard, 
WebML-generated interface of the vertical, populated by content spawning a large body 
of knowledge; but we also offer to the focused learners a guide, available on an interface 
that can be opened “aside” the main one, and that points to pages and contents published 
by the WebML-generated interface, according to a specific learning objective and user 
experience. 

The integration is loose and preserves the distinctive features of the two systems. In 
particular, some nodes and links in a UML-Guide state diagram point to content which 
is managed in the WebML e-learning vertical; therefore, the integration of UML-Guide 
with WebML requires UML-Guide adopting WebML concepts, such as page identifiers 
and content identifiers. In this way, concepts used as state names or as tagged values 
within UML-Guide are mapped to learning resources stored in the database generated 
from the WebML data model. 

In the resulting application, the user-specific adaptation occurs in UML-Guide. This 
separation of concerns represents an extreme solution, as it is possible to support person- 
alization [5] and adaptation [2] directly in WebML. However, the proposed solution is an 
example of how client-side computations, specified at high-level in UML, can integrate 
WebML-designed solutions. As such, this experiment can be replicated for many other 
applications and the focus on UML-Guide can pursue different objectives. 

Figure 5 describes the system architecture. The high-level WebML and UML-Guide 
specifications are mapped into XML-based internal representations, respectively built 
by the Code Generator component of WebRatio [4] and by the XMI [ 13] Generator of 
Poseidon. 

The WebML run-time component runs JSP templates (also embedding SQL), and 
uses XSL style sheets for building the application’s presentation. The XMI represen- 
tation of a UML-Guide drives a run-time adaptation engine, written in XSLT, which 
dynamically changes the content of the profile variables and produces the UML-Guide 
user interface. The WebML and UML-Guide interfaces are then composed and presented 
to the user. 

In this architecture, the main integration issue is concerned with the generation of 
WebML links “pointing” to the WebML-controlled portion of the application, to be 
addressed while building the UML-Guide interface. WebML links take the format: 



7 http : / /www . gentleware . com/ 
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Fig. 5. Architecture of the composed system. 



ApplicationURL/page_identif ier . do?ParameterList 

where page_identif ier denotes a WebML page and ParameterList is a list of 
tag-value pairs, in the form entity_id . attribute=parameter. Thus, UML-Guide state 
diagrams must be extended with tagged values to be used as pointers to WebML concepts. 

Figure 6 depicts an excerpt of state diagram extended with tagged values for WebML 
concepts, needed for computing WebML links. This work must be performed by UML- 
Guide designers, typically in the course of the transformations required for “implement- 
ing” UML-Guides starting from their high-level descriptions (as illustrated in Figure 3). 

For instance, Object Oriented Programming Concepts is a lecture. The cor- 
responding page name is LectureModules from WebML hypertext model. The entity 
used to store lectures in the WebML data model is L0. The title used as an attribute to 
identify the lecture is the same as the state name. Entry and exit actions are transformed 
if they send parameters in WebML links, as it is in the case of Relations To Code 
(where the parameter of the show method is replaced by specific WebML parameter 
&L0 . Title=Procedures). It is worth noting that, although in our example tagged val- 
ues for page and entity names are constant values, in more complex cases they can be 
specified as well as parametric selectors, so as to automatically retrieve their values from 
the WebML XML specification based on specific conditions. Also, more tagged values 
can be needed, for example for identifying content units IDs, in situations where the 
selection of pages is based upon the content units they include. 

Queries for retrieving OID’s of the WebML concepts and content are submitted 
through a specifically designed interface to the WebML run-time components. The in- 
terface consists of the two functions GetWebMLConcept (Type , Name) and 
GetWebMLRecordOID (Entity , Attribute, Value). 



5 Generation of the Integrated E-learning Application 

Figure 7 presents the user interface of the integrated application. The UML-Guide gen- 
erated map, obtained as a transformation of the UML state diagram depicted in Figure 3, 
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Fig. 6. Excerpt of the UML-Guide state diagram extended with tagged values representing WebML 
concepts. 



is on the left; the WebML application, generated from the specification of Figure 2, is 
on the right. While the WebML application has an arbitrary interface, which depends on 
content composition within pages and on the adopted presentation style, the UML-Guide 
interface has a given structure that includes the following elements: 

- Folder symbol — represents a composite information fragment composed by other 
(simple or composite) information fragments and links. The composition is visually 
represented by the plus/minus symbol, for showing/hiding enclosed items, and by 
the left hand indent of enclosed items. A content can be associated to each symbol. 

- Dashed box symbol — represents a composite information fragment, which has to 
be presented concurrently with other composite information fragments (the dashed 
boxes) depicted on the same level. 

- Document symbol — represents a simple information fragment; only links can be 
nested under it. 

- Arrow symbol — represents a link to another composite or simple information frag- 
ment; the arrow symbols can be nested under the folder when they represent different 
alternatives of suggested links starting from a particular document. Each arrow is 
associated with a content and a name of the corresponding target node. Also, the 
“/next” string is added to names of arrows representing guidance to the next fragment 
according to the course sequence. 

- Grayed background of nodes — represents the currently presented node, i.e., the 
position reached by a user in the navigation map. 

Presentation for the adaptive navigation support depends on the generator settings. 
For example, according to the traffic light metaphor, adaptive recommendations may 
be represented through different colors (green for nodes appropriate with respect to 
the current state of the user profile, red for not appropriate nodes, yellow for other 
situations — e.g. a node that has been already visited). Also, other metaphors might show, 
hide, or sort the nodes. 

Profile records are maintained at the client side. When users begin a new session, 
their profile is initialized from a client-side XML-based database. The navigation map is 
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Fig. 7. Visualization of the navigation graph for the Java e-lecture. 



manipulated at the client side as well. Javascript is used to implement the user interface 
control and user profile manipulation. The events generated by user actions on the user 
interface invoke profile adaptation actions, which possibly process and add new values 
to the user profile. They also trigger regeneration of the navigation map, according to 
the newly computed values. 

The navigation map responds to changes in user profile by changing recommendation 
annotations (e.g., changing colors of nodes or showing/hiding nodes). When specific 
requirements, for example those set by conditions in entry actions of states, are met, 
the WebML vertical adapts delivered content based on additional parameters that UML- 
Guide is able to send to the server-side application. 



6 Related Work 

Model-driven development of Web applications has been intensely investigated during 
last years [6, 1 0, 1 1 , 1 7] . WebML has been proposed for the model-driven design of “data- 
intensive" Web applications. Its distinguishing feature is that the proposed design method 
is also supported by XML- and Java-based technologies, which enable the automatic 
generation of the application code [4]. 

During last years, some approaches have been proposed for extending traditional 
development methods by means of features enabling one-to-one personalization [1,15, 
17]. The aim is to customize the applications contents and the hypertext topology and 
presentation with respect to user preferences and needs, so as to offer an alternative to 
the traditional “one-size-fits-all” static approach in the development of Web systems [1]. 
Some proposals cover the adaptivity of Web applications with respect to some other di- 
mensions characterizing the context of use (see [14] for a complete survey). WebML also 
offers some constructs for personalization. In particular, the application data schema can 
be extended through some entities representing user profiles and user access rights over 
the application content [5]. Recently, WebML has also been extended with some prim- 
itives for modelling context-aware applications, i.e., mobile, personalized applications 
that are also able to adapt to the current situation of use [2] . However, WebML, as well as 
the majority of Web personalization and adaptivity approaches so far proposed, manages 
personalization at server-side, and does not offer the alternative of managing user profiles 
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and personalization policies at client side. Conversely, the UML-Guide approach estab- 
lishes model-driven design for adaptive applications, by considering link level adaptation 
and content level adaptation at the client side, where adaptation is computed according 
to the UML design specifications. First, requirements are modelled as variation points 
with mandatory and optional features in application domain models [7]. Guard logical 
expressions and adaptivity actions are used in navigation specifications [9]. A rule based 
approach has been also employed in more open environment based on semantic web 
models [8]. 

7 Conclusions and Further Work 

This paper has shown the integration between WebML and UML-Guide; the proposed 
approach demonstrates that server-side and client-side technologies can coexist and that 
it is possible, for both of them, to use model-driven code generation techniques starting 
from high-level requirements, expressed in graphical form. The proposed application 
scenario augments an ‘e-learning” vertical so as to make it adaptable and personalisable. 

In our experiments, we moved especially user dependent functionality to the client 
side which allowed us to leave control over sensitive user data to the user. Users can decide 
on their own which information will be disclosed and which they will deny access to. 
On the other hand, this increases requirements for client-side tools to be able to interpret 
a database with information about a user and process it for purpose of adaptation and 
personalization. As client machines are usually less powerful, this might result in some 
lacks of performance. We will further investigate and experiment with the approach 
proposed to find a good balance between client-side and server-side processing. 

We regard this work as the first step of a deeper methodological inspection of the 
interactions between UML and WebML, and more specifically of the possibility of using 
state diagrams, which best represent the modeling of dynamic interfaces, for collecting 
the requirements that can naturally evolve into WebML specifications. The experiments 
described in this paper, and specifically the mechanisms for rendering state transitions 
as WebML links, will be extended and reused. In this paper we showed an integration of 
the two methods on an application where the state diagram is used to model interaction 
over information from one class. As a part of the deeper investigation, more complex 
applications will be studied where interaction between objects of several classes implies 
a need to use other behavioral techniques (collaboration and sequence diagrams) together 
with state diagrams. 

We are also planning an extension of the WebML CASE tool and of UML-Guide for 
providing automatic support to the integration of the two methods. 
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Abstract. This paper describes the personalisation services designed 
for self e-learning networks in the SeLeNe project. These include person- 
alised search results, view definition over learning object metadata, meta- 
data generation for composite learning objects and personalised event 
and change notification services. 



1 Introduction 

Life-long learning and the knowledge economy have brought about the need 
to support diverse communities of learners throughout their lifetimes. These 
learners are geographically distributed and have heterogeneous educational back- 
grounds and learning needs. 

The SeLeNe ( self e-learning networks) project is investigating the feasibility 
of using semantic web technology to support learning communities and match 
learners’ needs with the educational resources potentially available on the Web. 
SeLeNe relies on semantic metadata describing educational material, and is de- 
veloping services for the discovery, sharing, and collaborative creation of learn- 
ing objects (LOs). A SeLeNe will facilitate access to LOs by managing their 
metadata descriptions. In order to enable effective search for LOs in a SeLeNe, 
LO descriptions conform to e-learning standards such as IEEE/LOM (Learn- 
ing Object Metadata), and also employ topic-specific taxonomies of scientific 
domains such as ACM/CCS (Computing Classification System) or taxonomies 
of detailed learning objectives. LO schemas and descriptions are represented in 
the Resource Description Framework/ Schema Language (RDF/S), which offers 
advanced modelling primitives for the SeLeNe information space. 

The diversity and heterogeneity of the communities we envisage using Se- 
LeNes means that no single architectural design will be suitable to support all of 

* A longer version of this document that includes detailed examples is available from 
the SeLeNe project website: http://www.dcs.bbk.ac.uk/selene. 

N. Koch, P. Fraternali, and M. Wirsing (Eds.): ICWE 2004, LNCS 3140, pp. 215-219, 2004. 
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them. We have thus defined a service-based architecture that can be deployed in 
a centralised, mediation-based or peer-to-peer fashion, so the deployment option 
best addressing the needs of any particular learning community can be chosen 
(see [1]). Figure 1 illustrates the service architecture of a SeLeNe. The facili- 
ties that are the focus of this paper are provided by the User Registration, LO 
Registration, Trails and Adaptation, Event and Change Notification, and View 
services. 




Fig. 1 . SeLeNe service architecture 

The users of a SeLeNe will include instructors, learners and providers of LOs. 
When registering, users will supply information about themselves and their edu- 
cational objectives in using this SeLeNe, which forms the basis of their personal 
profile. LO authors maintain control of the content they create and will be free 
to use any tools they wish to create their LOs before registering them. We call 
such LOs, created externally to SeLeNe, atomic LOs. Users will also be able 
to register new composite LOs that have been created as assemblies of LOs al- 
ready registered with the SeLeNe. The taxonomical descriptions for these LOs 
can be automatically derived by the SeLeNe from the taxonomical descriptions 
of its constituent LOs and this process is discussed in Sect. 2. The SeLeNe 
will provide facilities for defining personalised views over the LO information 
space, discussed in Sect. 3, and also personalised searching facilities, discussed 
in Sect. 4. Provision of automatic change detection and notification facilities al- 
lowing personalised notification services depending on users’ profiles is discussed 
in Sect. 5. 
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2 Registration of LOs 

Registration of a LO with a SeLeNe consists of providing a metadata description 
including the URI of the LO. An important feature of SeLeNe’s LO Registra- 
tion service will be the ability to automatically infer the taxonomical part of 
the description for a composite LO o, from the taxonomical descriptions of its 
component LOs 0\, . . . , o n . This description, which ‘summarises’ the taxonom- 
ical descriptions of the parts of o, is called the implied taxonomical description 
(ITD) of o. This is used to augment the publisher taxonomical description (PTD) 
- the set of terms supplied by the LO’s provider (which can be empty) - to derive 
the final taxonomical description used to register the composite LO. 

The ITD of a composite LO o expresses what its parts have in common - 
only terms reflecting the content of all parts are included in the ITD. A LO may 
thus generate different ITDs depending on what its ‘companion’ parts are within 
a composite LO. 

The ITD of a composite LO o composed of parts Oi , . . . o n with descriptions 
D i , . . . , D n is computed by a simple algorithm that takes the cartesian prod- 
uct of D\, . . . , D n , computes the least upper bound of each n-tuple and then 
‘reduces’ the resulting set of terms by removing all but the minimal terms ac- 
cording to the subsumption relation A between the terms of the taxonomy. The 
overall taxonomical description of a LO o is computed by another simple algo- 
rithm: if o is atomic then its taxonomical description is just its PTD. Otherwise 
its taxonomical description is recursively computed from its PTD and the taxo- 
nomical descriptions of its constituent parts. Readers are referred to [2] for more 
details of both algorithms. 

3 Declarative Queries and Views 

Finding LOs in a SeLeNe will rely on RQL [3], a declarative query language 
offering browsing and querying facilities over the RDF /S descriptions that form 
the SeLeNe information space. 

As well as the advanced querying facilities provided by an expressive RDF /S 
query language such as RQL, personalisation of LO descriptions and schemas 
is also needed. For instance, a learner might want LOs presented according to 
his/her educational level and current course of study. To enhance the SeLeNe 
user’s experience we need the ability to personalise the way the information space 
can be viewed, by providing simpler virtual schemas that reflect an instructor’s 
or learner’s perception of the domain of interest. This will be done by using 
the RVL language [4], which provides this ability by offering techniques for the 
reconciliation and integration of heterogeneous metadata describing LOs, and 
for the definition of personalised views over a SeLeNe information space. 

One of the most significant features of RVL is its ability to create virtual 
schemas by simply populating the two core RDF/S metaclasses Class and Prop- 
erty. A SeLeNe user can then easily formulate queries on the view itself. RVL can 
also be used to implement advanced user aids such as personalised navigation 
and knowledge maps. 
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4 Trails and Query Adaptation 

Personalisation of query results will rely on the personal profile. This is an RDF 
description of the user including some elements from existing profile schemas 
(PAPI-Learner and IMS-LIP), some of our own elements (we have defined RDF 
schemas for expressing competencies, learning goals and learning styles), and a 
history of user activity that will allow the profile to adapt over time. 

Although the underlying query mechanism in SeLeNe is RQL, users will 
mostly search for LOs using simple keyword-based queries. Search results will 
be personalised by filtering and ranking the LOs returned according to the in- 
formation contained in the user’s personal profile. This is done by the Trails and 
Adaptation service, which constructs personalised RQL queries for execution 
and generates personalised rankings of query results by matching the personal 
profile against relevant parts of the LO descriptions. 

SeLeNe will give the user the option of having their query results presented as 
a list of trails of LOs, suggesting sequences of LO interaction. These trails will be 
automatically derived from information contained in the LO descriptions about 
the semantic relationships between LOs. We have defined an RDF representation 
of trails as a sub-class of the RDF Sequence (a sequence of LOs) with two 
associated properties, name and annotation. 

5 Event and Change Notification 

We provide SeLeNe’s reactive functionality by means of event-condition-action 
(ECA) rules over SeLeNe’s RDF/S metadata. This includes features such as: 

— automatic propagation of changes in the description of one resource to the 
descriptions of other, related resources (e.g., propagating changes in the tax- 
onomical description of a LO to those of any composite LOs depending on it; 
propagating changes in a learner’s history of accesses of LOs to the learner’s 
personal profile; automatic generation and update of ‘emergent’ trails); 

— automatic notification to users of the registration of new LOs of interest to 
them; 

— automatic notification to users of changes in the description of resources of 
interest to them. 

SeLeNe’s RDF ECA rules will be automatically generated by the higher-level 
Presentation and Application services, and may be resource-specific or generic 
(i.e., they might or might not reference specific RDF resources). 

Peers that support the Event and Change Notification service will run an 
ECA Engine consisting of three main components: an Event Detector, Condi- 
tion Evaluator and Action Scheduler. The Query service is invoked firstly by the 
Event Detector to determine which rules have been triggered by the most recent 
update to the local description base, and again by the Condition Evaluator to 
determine which of the triggered rules should fire. The Action Scheduler gener- 
ates a list of updates from the action parts of these rules, which are then passed 
to the Update service for execution. We refer the reader to [5] for a description 
of the syntax and semantics of our RDF ECA rules, and some examples. 
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6 Concluding Remarks 

This paper has described several novel techniques for providing the personalisa- 
tion services of self e-learning networks. The novel aspects of SeLeNe compared 
with other related systems 1 include: 

— collaborative creation and semi-automatic description of composite LOs; 

— declarative views over combined RDFS/RDF descriptions (i.e. over both the 
LO descriptions and their schemas); 

— personalised event and change notification services; 

— automatic generation of trails of LOs from their descriptions. 

We are currently implementing and integrating the components of SeLeNe. 
A number of open issues remain. Firstly, there is as yet no standard query or 
update language for RDF, although we believe that the RQL, RVL and RDF 
ECA languages we have described here provide sound and expressive foundations 
for the development of such standards. Secondly, whatever standards eventually 
emerge for such RDF languages, if ECA rules are to be supported on RDF 
repositories then the event sub-language for RDF ECA rules needs to be designed 
so that it matches up with the actions sub-language; another important open 
area is combining ECA rules with transactions and consistency maintenance 
in RDF repositories. Thirdly, our algorithms for personalised ranking of query 
results need to be empirically evaluated. Finally, the design of user interfaces 
enabling easy and intuitive access to SeLeNe’s advanced personalisation services 
will be crucial - users will need to be shielded from the complexities of RDF and 
the RQL, RVL and ECA languages, and also from the complex taxonomies of 
topics, competencies and goals in use by the system. 
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Abstract. Despite recent advances in wireless and portable hardware 
technologies, mobile access to the Web is often laborious. For this reason, 
several solutions have been proposed to customize Web pages to mobile 
access. In this paper, we describe a wrapper system that targets the 
personalization of Web pages to mobile devices. A personalization is a 
subpage of a given web page containing just the information that mobile 
users would like to access using a PDA. 



1 Introduction 

Mobile access to the web is often laborious, since most of the web content was not 
produced for mobile devices. In the last few years, several solutions have been 
proposed to customize and adapt Web pages to mobile access. In general, these 
solutions can be disposed on three categories: (i) redesign of web sites for use in 
mobile devices; (ii) usage of transcoders, i.e., proxies responsible for reformatting 
web pages, such as Digestor [2] and PowerBrowser [3]; (iii) usage of wrappers, 
i.e., programs that automatically retrieve and extract particular content from 
Web documents, such as Lixto [1], Web Views [4] and Smartview [6]. 

In this paper, we describe a wrapper system that targets the personalization 
of Web pages to mobile computing devices. A personalization is a subpage of a 
web page that contains just the information a given user would like to access 
using a PDA. The proposed personalization system - called PWA - meets the 
following requirements: (i) it supports personalization of standard web pages, 
despite if they are well-formed or not; (ii) it supports personalizations that are 
robust to common changes in Web pages; (iii) it supports the creation of person- 
alizations using a GUI, requiring no pattern matching language programming. 

The rest of this paper is organized as follows. In Section 2 the architecture of 
the system is described, along with its algorithms and data structures. Section 3 
presents the first results we have obtained in the creation of personalizations and 
Section 4 concludes the paper. 
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2 The PWA System 

The PWA system has two basic modules: the personalization definition system 
and the personalization server. These two modules are described next. 

2.1 Personalization Definition System 

The personalization definition system is in charge of creating extraction expres- 
sions, i.e., expressions that when applied to a web page return the corresponding 
subpage. 

User Interface: In the PWA system the creation of extraction expressions 
happens in a desktop computer. The following steps are required: 

1. The user should accesses the web page he want to personalize using a stan- 
dard browser. 

2. Using the mouse, the user should select the components of the page he wants 
to access later in his mobile device. The selection process is identical to the 
one used to select texts in standard editors. 

3. The user should copy the selected text to the clipboard. 

4. The user should start the personalization definition system and choose the 
option New Personalization in its main menu. From the text saved in the 
clipboard, the system will generate an extraction expression and will present 
the personalization associated to this expression to the user. 

5. If the user is satisfied with the personalization, he should choose the Export. 
Personalization option in the main menu. The personalization will be saved 
in the personalization server. 

This process is fully based on standard browsers and in the built-in copy- 
and-paste facility provided by common window systems. 

Data Structures: In order to create an extraction expression, the personaliza- 
tion definition system relies on the following data structures: (i) list of tags, i.e., 
a linear list whose nodes store information about the tags and the text of the 
target web page; (ii) buffer, i.e., an array containing just the raw text of the 
target web page, (iii) mapper, i.e., a function from Int — > Int. A mapping like 
i — > k means that the text stored in the i-tlr position in the buffer comes from 
the k- th text node in the list of tags. 

Figure 1 describes the previous data structures for the fragment of a simple 
web page. The design of the PWA system was inspired in the MVC architecture 
[5]. The previous data structures represent the model in the MVC architecture, 
the browser represents the view and the clipboard the controller. 

Extraction Expression: The algorithm used to create the extraction expres- 
sion has as input value the text T that the user has selected and copied to the 
clipboard. First, T is matched against the buffer, resulting in the range of buffer 
indexes [ib,jb\. If T occurs more than once in the buffer, the intervention of the 
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Fig. 1. List of Tags, buffer and mapper structures 



user is requested to select the appropriate match. Next, using the mapper func- 
tion, the range [ib,jb\ is translated to indexes [i,j\ in the list of tags containing 
the nodes where T is stored. The sublist of the list of tags defined by indices [iff] 
is called S. The extraction expression is composed by the tuple: ( N , U, T, i, j, S) , 
where N is the name given to the expression by the user and U is the URL from 
the associated web page. This tuple is exported to the personalization server. 



2.2 Personalization Server 

The PWA server performs the following tasks: (i) it receives requests for 
personalizations from mobile users; (ii) it retrieves the web page associated to 
the requested personalization; (iii) using the extraction expression previously 
informed by the personalization definition system, it extracts the requested 
personalization from the retrieved web page; (iv) it sends the personalization to 
the device of the mobile user. 

Data Structures: Besides the data structures described on Section 2.1, the 
personalization server makes use of an extra structure, called Skeleton. The 
skeleton is a copy of the list of tags except by the fact that tags that only provide 
formatting information (such as <B>, <I> etc), programs (such as <APPLET>, 
<SCRIPT>) and images (such as <IMG>) are purged. Tags not presented in the 
skeleton are called volatile. Volatile tags are often removed and inserted in 
web pages. Therefore, expression extractions that consider them can easily break. 

Extraction Algorithm: The extraction algorithm has as input value the ex- 
traction expression (N, U, T, iff, S) previously saved in the personalization server 
by the personalization definition system. The algorithm has the following steps: 

1. The page associated to URL U is retrieved and the buffer, mapper and list 
of tags data structures described on Section 2.1 are created for this page. 
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2. The text T is matched against the buffer. If this match succeeds and the 
matched indexes are the same indexes [i, j] saved in the extraction expres- 
sion, the personalization is the sublist S. This happens when the associated 
web page is static (i.e., its content does not change with the time) and its 
structure has also not changed since the time the extraction expression was 
created. 

3. If the previous step fails, this means that the page has changed. Then, the 
idea is to verify if the change is restricted to the content of the person- 
alization, i.e., if the tags (or the structure) of the personalization remains 
unchanged. Therefore, the sublist S is matched against the full list of tags 
created in the step 1. A sublist S' located in the indexes [if j'] of the list 
of tags is the returned personalization if it matches S. In case of multiple 
matched sublists, we chose the one closest to the original position of S, i.e., 
the match where \i — i'\ is minimum. 

4. If the previous step fails, this means that both the content and the structure 
of personalization have changed. The idea is to verify if the changes were 
restricted to the insertion or removal of volatile tags. First, the skeleton 
data structure is created. Next, the volatile tags of S are removed and a 
new sublist S' is created. This new sublist is matched against the skeleton. 
A sublist S" located in the indexes \i" ,j"] of the skeleton is the returned 
personalization if it matches S'. In case of multiple matches, we chose the 
one closest to the original position of S, i.e., the match where | i — i"\ is 
minimum. 



Table 1. Experimental Results. Column Success contains the number of days the 
personalization was extracted successfully. Column Changes Inside contains the number 
of days the original page has undergone changes inside the area of the personalization. 
Column Changes Outside contains the number of days the original page has undergone 
changes outside the area of the personalization. 



Site 


Success 


Changes Inside 


Changes Outside 


Bloomberg (Insight & Commentary) 


13 


13 


13 


Bloomberg (Market Snapshot) 


13 


0 


13 


Yahoo Sports 


13 


13 


13 


Google News 


12 


13 


13 



Suppose that P is the personalization returned by the previous algorithm. 
This personalization is a list containing tags and text. Most of the time, this 
raw list cannot be directly transmitted to mobile devices, since it does not 
contain appropriate context tags. For example, maybe the first tag of P is 
a <TD>. However, a <TD> can never appear without an enclosing <TR>, and 
this last cannot exist without an enclosing <TABLE>. Thus, the personalization 
P is augmented with appropriate context tags before it is sent to the mobile user. 
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3 Experimental Results 

In order to validate the PWA system, we have created personalizations associ- 
ated to the following sites: www.bloomberg.com (two personalizations, compris- 
ing the Market Snapshot and the Insight & Commentary sections of the page), 
news . google . com, (one personalization comprising the Top Stories section of the 
page) and sports.yahoo.com (one personalization comprising the top headline 
section). We have accessed these personalizations during 13 days, in order to 
verify the robustness of the proposed extraction algorithm. Table 1 summarizes 
the results we have obtained in this experiment. 

As described in Table 1, the personalizations for the Bloomberg and Yahoo 
Sports web page have remained working for the whole duration of the exper- 
iment. The only personalization that has broken during the test was the one 
associated to the Google News page. This personalization has not produced the 
expected content in the 13tlr day of the experiment. The reason was a radical 
change in the structure of the personalization. 

4 Concluding Remarks 

We consider that the PWA system presents the following contributions: 

— The definition of personalizations by end-users is fully based on a GUI, 
integrated to a standard browser and to the system clipboard. This GUI is 
structured according to the well-known MVC model. 

— Our first experiments have shown that the proposed extraction algorithm is 
robust to a considerable types of changes. Particularly, it is fairly robust to 
changes outside the personalization area. Moreover, its is robust to changes 
inside the personalization if they affect only volatile tags. 

As part of future work, we will focus on the definition of personalizations 
including different and non-continuous sections of the same web page. Finally, 
more extensive experiments will be performed. 
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Abstract. In this article we describe our experience in the development of a 
personalizable dissemination model for the Miguel de Cervantes Digital 
Library's Web-based newsletter-service, which combines adaptive with 
adaptable personalization techniques, being capable or ranking news according 
to navigation-inferred preferences and then filter them according to a user-given 
profile. We explain how Web engineering design techniques have been applied 
to make that service evolve into a more adaptive personalization approach 
obtaining more effective results. The work is presented in the context of the 
OO-H [5] web design method. 



1 Introduction 

The Miguel de Cervantes Digital Library (MCDL) of the University of Alicante is the 
biggest electronic publishing project in Spain, and perhaps the biggest digital library 
of Spanish texts on the Internet, currently with more than 10000 entries in its 
catalogue. One of the goals of the MCDL is to act as a communication channel for 
the academic community. In this sense we have implemented a number of 
communication services, and we try to maintain a permanent communication with our 
readers. It is obvious that the personalization of these services is a key issue in the 
management of the DL in order to get a better use. 

The idea of giving the user the impression of interacting with the application by 
means of a dedicated interface, specifically tailored to the user’s needs and 
preferences is a “must” in current Web development. Content personalization systems 
select, adapt and generate contents according to user models that define information 
needs. Dissemination models help to achieve this goal. In a dissemination model 
environment, users subscribe to an information dissemination service by submitting a 
set of personalization settings that describe their interests, which are usually called 
profiles. Then they passively receive information filtered according to such profiles. 
Enhanced dissemination models allows the sender of such information to get a 
feedback of the users interests, which may evolve with time, or to get a feedback of 
the users opinions on the information received. In this context, user profile and 
preferences can be acquired in three ways [1]: be set by the application designer by 
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means of stereotyped techniques, be given by the users themselves by means of 
interview techniques (this includes input forms) or be automatically inferred by the 
application based on user activity ( observation techniques). The applications of the 
second type are also referred to as “ adaptable ” while the third type are “adaptive” , 
and when the origin of personalization information is a system event we talk about 
“ proactive ” applications [2, 3], 



2 Personalization of the Newsletter Service: Ranking and Filtering 

In the first stage, our profile was very simple and allowed only for the choice of one 
or more possible thematic newsletters out of five [4], In the second stage of 
development, we have remodeled the personalization approach to both offer a finer 
granularity (more detailed personalization choices) and also obtain explicit (adaptable 
approach) and implicit (adaptive approach) feedback from users which allows for 
dynamic changes in personalization settings and will also provide, as said before, 
useful information on user navigation habits and preferences. The user model we have 
implemented for newsletters automatically and transparently incorporates information 
gathered from user navigation (adaptive part). In addition, the user can set-up some 
filtering restrictions and customization preferences when registering for this service 
(adaptable part). The final model is based both on implicit interests on certain digital 
library sections (information gathered during navigation) and on explicit preferences 
that compose the user profile. 

News are classified by category and subject-matter. Categories are: new publications 
(new digital resources), future publications, new sections, chat announcements, call 
for papers, suggestions from our departments, letters from readers, visits of important 
people, contests, and the remainder are classified as general news. Subjects or matters 
are derived from the actual thematic structure of the DL. Each theme section or 
subcollection generates a subject-matter, as for instance: Latin-American literature, 
humanities research, history, children's literature, theatre, interactive services of the 
DL, computers and humanities, critical studies, tribute to Hispanists, Argentine 
Academy of Letters, PhD theses, movies, magazines/journals, recently printed books, 
law, and many more. This allows for a very fine granularity. Every time the user 
clicks on an entry of the newsletter’s table of contents, the Web page jumps to a 
single piece of news, and the server increments in one the corresponding category and 
subject-matter counters. Only one category but multiple subjects can be assigned to a 
piece of news. Relative access frequencies can be computed for categories and for 
subjects. Then news can be given a ranking value for a given user for a given news- 
reading session, which is calculated as the sum of subject frequencies of the subjects 
corresponding to a given piece of news, multiplied by the frequency of its category. 
For instance, if a user has an access frequency of 0.3 for the “new-publications” 
category, and, 0.1 for the “history” and 0.2 for the “PhD-lheses” subjects, then a piece 
of news announcing the publication of a PhD thesis on history will weight (0. 1 + 0.2) 
0.3 = 0.09, and will be ranked accordingly. On registration, the users can specify 
Boolean constraints for categories and subjects, saying which ones should be sorted 
out and which should be displayed. This profile can be modified by the user. 
Newsletters are accessed through a monthly index, where news are ordered first by 
category and then by subject, according to the dynamically computed ranking. But not 
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all the ranked news appear, they are filtered according to the user explicit profile. The 
first N 1 ranked entries that pass the filter are shown openly, and the rest appear as a 
collapsed "more news” button. 



3 Adapting the MCDL Using the OO-H Personalization 
Frame-Work 

The OO-H ( Object Oriented Hypermedia ) method [5] is a generic model, based on 
the object oriented paradigm that provides the designer with the semantics and 
notation necessary for the development of web-based interfaces. Web design 
modelling is achieved by means of the two complementary views, namely (1) the 
Navigational Access Diagram, that enriches a standard UML class diagram with 
navigation and interaction properties, and (2) the Abstract Presentation Diagram that 
gathers the concepts related both to structure of the site and specific presentation 
details respectively. OO-H also supports dynamic personalization (described in [6]), 
allowing the designer to better tailor the site to the particularities and needs of the 
individual user. This is done by means of a personalization framework that is a part of 
the model. That framework can be instantiated by the web designer, and connected to 
any OO-H based site to empower it with personalization support for (individual) 
users. The framework can be divided in two parts: (1) the user model, which allows 
to store the beliefs and knowledge the system has about the user, and (2) the 
personalization model, which is used to specify the personalization strategy for the 
different groups of users. 

In the context of the MCDL, we have used this approach to personalize the 
newsletters. In Figure 1 we can see the user model for the MCDL case study. 

The user model represents the variable part of the conceptual model. We have two 
association classes in which we store by subject-matter and by category the number 
of times that the user consults a piece of news. We also store the user preferences in a 
class that the user can change when desired. 

The personalization model allows the designer to define a collection of rules that can 
be used to define a personalization strategy for a user or group of users. The rules are 
Event-Condition-Action [6] rules: they are triggered by a certain event (e.g. a 
browsing action, the start of a session) and if a certain condition is fulfilled, the 
associated action is performed. Figure 2 shows the NAD corresponding to this 
modeling example. The NAD’s entry point is a link to a user-connection parameters 
form to access the system (login, password). Registered users are shown a link to 
access the newsletter. The personalization rule is attached to it. When the user clicks 
on it, a table of contents appears, showing the ranked and filtered list of news. To 
determine the user interest in a piece of news, we have the Acquisition rule that is 
attached to the “consult” service link. In this rule we’ll acquire the number of times a 
user consults a piece of news and its related subject-matters and category, and using 
these two attributes together with the user-set preferences profile, we rank and filter 



1 N is a user given parameter. 
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Fig. 1 . User and Domain models 



Fig. 2. NAD for the newsletters system 



the news according to the user’s interests. The ranking is computed with the above 
described algorithm. Later, this NAD is compiled to obtain the XML specification 
that is interpreted by a rules engine to give personalization support to our application. 
That compilation process produces 2 types of XML specs acquisition rules and 
personalization rules that are presented next. 

Acquisition Rule: Below the dotted line of figure 2, we can see the part of the NAD 
diagram that holds the rule that stores the preferences information (acquisition rule). 
Every time a user consults a piece of news (event that triggers the rule), the 
corresponding subject-matters and category counters must be incremented. To store 
this information (in the user model), we need to check whether the subject-matter and 
category of the piece of news the user has consulted are the same we are going to 
update in the User model. With these data we can predict the user preferences. The 
XML specification of the acquisition rule defined by our system follows: 

<rule type= "acquisition " name= " StorePref erences " > 

<params> 

<param name="idsbm" 

value= "session. registereduser . newsletter .news . subjectmatter . id" /> 
<param name= " idcat " 

value= "session . registereduser .newsletter .news . category. id" /> 

</params> 

<event type="MethodInvocation" link= " Consul t " /> 

<condition " idsbm=newstittle . news . subj ectmatter . id and 
idcat=news tittle .news . category, id" / > 

<action value= " session. registereduser . sbmcount . count++ 
and session. registereduser. catcount.count++" /> 

</rule> 



We use the subject-matter and category identifiers of the piece of news as parameters 
of the rule, to simplify the expressions. 

Personalization Rule: Above the dotted line of figure 2, we can see the part of the 
NAD diagram that holds the rule that models this requirement (personalization rule). 
When the link View Newsletter is activated the rule is triggered. In this case we have 
navigation personalization, because we have to sort the links the user is going to see. 
We have the two attributes as parameters (stored in the user model by the acquisition 
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rule). The personalization event indicates that the link View Newsletter must be 
active. When this link is activated, the action will be executed: the news are sorted 
and shown. 

crule type= "personalization: navigation" name= "Recommendations " > 
<params> 

<param name= " sbmcount " value=" 
session . registereduser . sbmcountcount " /> 

<param name="catcount" value=" 
session . registereduser . catcountcount " /> 

</params> 

<event type= "Navigation" link="View newsletter " /> 

<action type="Sort" link= "news tittle" byvall=" sbmcount" 
byval2="catcount" ORDERS "DESC" /> 

</rule> 



4 Conclusions 

Concerning the dissemination model for these newsletters, we have enhanced the 
granularity by offering more detailed personalization options, which allow us to rank 
the news based on preferences gathered from user navigation (observation model). 
The design of this solution was performed according to the OO-H Model. This 
technology can significantly increase the productivity at the time of developing Web 
applications. The MCDL, with this effort, struggles to fulfill its objective of spreading 
research knowledge to the global academic community through the Web. Our goal is 
not only the mere publication of research work, but to build a rich and open 
communication channel for the global scientific community. The newsletter service 
described here plays a key role in this communication effort. 
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Abstract. In order to support development of consistent and secure 
Web applications, we have designed a number of Web application gen- 
erators. These generators can be classified into two types of approaches: 
annotation approach and diagram approach. In this paper, we try to 
make the roles of these generators clear, and compare two approaches 
in terms of target applications, developing processes and target users. 
While both approaches are powerful and flexible enough to construct 
typical Web applications efficiently, we may select the most appropri- 
ate generator according to the characteristics of the application and the 
developing process. 



1 Introduction 

Today, Web applications such as database query systems and transaction sys- 
tems are widely used especially on the Internet. The development of such ap- 
plications, however, requires much cost and experience of developers because of 
the complexity of security checks and session management, which are unique 
to Web applications. In order to support development of consistent and secure 
Web applications, we have designed a number of Web application generators 
which generate source codes necessary to execute the application [1,2, 3, 4, 5, 6, 
7,8,9]. Web application generators encapsulate the complexity unique to Web 
applications and make developers concentrate on the business logic of the appli- 
cation. 

Our generators can be classified into two types of approaches. The first is 
annotation approach, which concentrates on input data and embedded values 
on each Web page [1,2]. Developers first compose Web page templates and give 
declarative annotations to them. And then the generator generates procedural 
program codes from Web page templates with annotations. The second is dia- 
gram approach, which concentrates on data-flow relationships in the application 
[3, 4, 5, 6, 7, 8, 9]. Developers first compose diagrams which describe overall data- 
flow relationships among Web components such as Web page templates, server 
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side programs and databases. After developers select appropriate program tem- 
plates and components, a generator can generate executable program codes from 
the diagrams. 

Each approach has assumptions and roles in developing processes. Developers 
can select the most appropriate generator according to the characteristics of the 
application and the developing process. In this paper, we try to make the roles 
of these generators clear, and compare the two approaches in terms of target 
applications, developing processes and target users. We also give discussions on 
our future work based on this comparison. 

The organization of the rest of this paper is as follows. In section 2 and section 
3, we describe annotation approach and diagram approach, and give examples 
of our generator systems based on each approach. In section 4, we compare two 
approaches in terms of target applications, developing processes and target users. 
In section 5 we give related work. We finally give future work and concluding 
remarks in section 6. 

2 Annotation Approach 

2.1 Basic Idea of Annotation Approach 

From the viewpoint of user interfaces, we can consider general Web applications 
as transitions between Web pages just like ordinary Web pages that have no 
server side programs. In most Web applications, the greater part of each Web 
page template is static, and a part of the template is dynamic where actual 
values are embedded by server side programs. Based on this idea, we present an- 
notation approach to automatic construction of Web applications. In annotation 
approach, we describe dynamic part of Web page templates as declarative anno- 
tations. More precisely, the following steps are taken in the generation method. 

1. We first construct Web page templates for intended Web applications. Web 
page composers and other support tools may be used to visually compose 
Web page templates efficiently. Dynamic part of the templates, where val- 
ues are embedded at run time, may be described as special characters to 
distinguish from static part. 

2. We give annotations to dynamic part of Web page templates. Our anno- 
tations are declaration of data processing which is executed on the server 
side. Their tasks are mainly related to data-flow relationships among Web 
components as follows. 

— For input data checking, such as checking acceptable types and length 
of input values and constraints between them. 

— For session management, such as checking the beginning and the end of 
the session. 

— For database handling, such as the access to database management sys- 
tems using queres. 

— For communications with external programs, such as invocation of Web 
services, EJB components and other applications. 
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Fig. 1. The architecture of A- Web system 



3. From Web page templates with annotations, a Web application generator 
automatically generates source codes of server side programs. Generated 
programs have consistency and the standard level of security because the 
generator checks transitions and parameters between Web page templates, 
and generates additional codes to prevent inconsistency. 



2.2 Generator: A- Web System 

Based on annotation approach, we designed and implemented a prototype gen- 
erator called A- Web system which generates Web applications from Web page 
templates [1,2]. While our current prototype can generate only CGI programs, 
we may be able to generate Web applications based on other architecture by re- 
placing part of the generator because annotation approach itself is independent 
of specific architecture. 

The architecture of A- Web system is shown in Fig.l. A- Web system consists 
of two parts: an annotation editor and a Web application generator called D-Web 
system. 

Web page templates. Before using A- Web system, Web page templates 
should be prepared as input of the system. We can compose Web page tem- 
plates using visual composers and other support tools because A- Web system 
requires ordinary XHTML documents. Dynamic part of the templates should 
be represented by special characters ${scope.valiable} . Variable is a name to 
distinguish from other variables among the application and should be unique 
in each scope. Because the special characters are ordinary strings, not exten- 
sion of HTML tags, common tools can deal with these templates. Thus it is 
unnecessary for us to modify the source codes directly. 

Fig. 2 shows an example of Web page templates of a simple member registra- 
tion system. This application first requires users to input id and password 
which the user chooses, a name, an email address and so on at a Web page 
’Registration’. If the id is already registered or the input data don’t satisfy 
specified conditions, a Web page ’Error’ is generated. Otherwise a Web page 




Comparison of Two Approaches for Automatic Construction 



233 




Fig. 2. An example of Web page templates 



’Confirmation’ is generated. After the confirmation, the registration becomes 
definite. 

An annotation editor. An annotation editor is a part of A- Web system, which 
is an editor to annotate Web page templates and convert them to D-Web 
source documents. When the editor gets Web page templates, it analyzes the 
templates, points out where we can give annotations and allows us to give an- 
notations visually. In our prototype, we introduce five types of annotations: 
input check, constraint, session, SQL and SOAP. 

An input check annotation defines conditions for acceptable users’ in- 
put. If the input is strings, we can define the length and types of ac- 
ceptable characters. If the input is numbers, we can define a range of 
acceptable numbers. 

A constraint annotation defines relations which must be satisfied among 
input data. 

A session annotation defines the behavior of session management for each 
template. When a user accesses a Web page with a session annotation 
’begin’, the program starts the new session. In the case of ’check’, the 
program checks whether the session is valid or not. In the case of ’end’, 
the program terminates the session. 

A SQL annotation describes SQL statements to access database manage- 
ment systems. We can deal with database transactions. 

A SOAP annotation describes statements to invoke external Web ser- 
vices using SOAP protocol. 

Fig. 3 shows an annotation editor in A-Web system. It analyzes Web page 
templates and adds hyperlinks to special annotation pages automatically. 
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<html> 

<head> 

<title>Registration</title> 

<session type="begin" error= " Error .html"/> 

</head> 

<body> 

<hl>Registration</hl> 

<form action="Conf irmation.html" method="Post"> 
id:<input type="text" name="id" length=" [6, 10] "/> 

omit following parts 

</body></html> 

Fig. 4. An example of D-Web source codes 



Fig. 3 shows ’Registration’ page, a session annotation page and an input 
annotation page. Fig. 4 shows an example of D-Web source codes, which are 
generated by the annotation editor from a Web page template ’Registration’ 
and its annotations. D-Web source codes have a number of extended tags. 

D-Web system. D-Web system is a part of A- Web system, which gets D-Web 
source documents and generates source codes of Web applications. In our 
prototype, D-Web system generates XHTML documents and CGI programs 
written in Perl. To keep consistency of generated applications, D-Web system 
analyzes all variables in Web page templates and transitions between Web 
pages, and gives methods to pass the values between Web pages correctly. To 
keep the standard level of security, all generated programs have additional 
codes as follows. 

1. After receiving input data, each program checks whether the user comes 
from correct Web page. 
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2. Then each program checks whether user’s session id is valid. 

3. Then each program checks whether input data are correct according to 
input check annotations. 

If there is at least one error, the user’s request is redirected to an error page. 

3 Diagram Approach 

3.1 Basic Idea of Diagram Approach 

From the viewpoint of data-flow relationships among Web components such as 
Web pages and programs, we can consider Web applications as applications based 
on pipes and filters architecture. A filter corresponds to a server side program 
and a pipe corresponds to a Web page which passes data between programs. 
Based on this idea, we present diagram approach to automatic construction of 
Web applications. In diagram approach, we first compose diagrams describing 
overall behavior of the application and select general-purpose templates and 
components to generate executable applications. More precisely, the following 
steps are generally taken in the generation method using this approach. 

1. We first compose directed graphs whose nodes represent Web components 
such as Web page templates, server side programs and databases, and whose 
edges represent data-flow relationships among the components. Most of our 
generators use diagrams called Web transition diagrams which we designed 
for the above purpose. 

2. All generators based on diagram approach have predefined program tem- 
plates and components which are independent of specific domains of ap- 
plications, for example, for purposes of database manipulations, sending 
electronic mails. Referring specifications of these programs, we give corre- 
spondence between nodes in diagrams and the programs, and give values of 
parameters of them. 

3. From the descriptions of diagrams and values of parameters, a generator 
automatically generates Web page templates and source codes of server side 
programs. Generated programs have consistency and the standard level of 
security, because the generator checks consistency of the diagrams, checks 
correspondence between diagrams and predefined programs, and then gen- 
erates additional codes to prevent inconsistency. 



3.2 Generator: T-Web System 

Based on diagram approach, we designed and implemented a prototype genera- 
tor called T-Web system which generates Web applications from directed graphs 
called Web transition diagrams [3, 4, 5, 6, 7, 8, 9]. While we have prototypes to gen- 
erate CGI programs, JSP/Servlet programs and ASP programs respectively, we 
may be able to generate applications based on other architecture by replacing 
part of the generators because diagram approach itself is independent of specific 
architecture of Web applications. 
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Web-Based Transaction System 



Fig. 5. The architecture of T-Web system 



In this section, we explain the basic architecture of T-Web system on the 
basis of the generator for JSP/Servlet-based applications. The architecture of 
T-Web system is shown in Fig. 5. T-Web system consists of two parts: a Web 
transition diagram editor and a Web application generator. 

Web transition diagrams. In T-Web system, we describe overall behavior 
of intended application as directed graphs called Web transition diagrams. 
Basically, Web transition diagrams consist of four types of nodes and two 
types of links as follows. 

A fixed Web page node is a static Web page which can be reached by 
a certain URL. Its notation is a rectangle with its name, whose line is 
thick. It may have a number of page elements such as hyperlinks and 
input fields inside the rectangle. 

A output Web page node is a dynamic Web page which is generated by 
a server side program. Its notation is a rectangle with its name, whose 
line is thin. Like a fixed Web page node, it may have a number of page 
elements. 

A processing node is a server side program which is activated by users’ 
requests to perform processing. Its notation is an oval with its name. 

A database node is a relational database table in a database server. Its 
notation is a cylinder with its name. The schema of the table is repre- 
sented by a list of names between ’{’ and 
A page transition link is a hyperlink relationship between Web pages. Its 
notation is a directed line. 

A data-flow link is a data-flow relationship among Web components such 
as Web pages, programs and database tables. Its notation is a directed 
line with a blocked line. 

Each generator may have some extension of Web transition diagrams accord- 
ing to the target architecture. Fig. 6 shows an example of Web transition dia- 
grams of a simple member registration system which is the same application 
as given in section 2. 
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Fig. 6. An example of Web transition diagrams 



A Web transition diagram editor. A Web transition diagram editor is a 
part of T-Web system, which is an editor to support composition of con- 
sistent Web transition diagrams. It allows users to do all operations visually. 
We compose Web transition diagrams as follows. 

1. We start up the editor and the editor receives available program tem- 
plates and components automatically. 

2. We draw Web transition diagrams by selecting a node from a list, putting 
it on a drawing field and arrange nodes. A links is given by selecting the 
target node from a list so that we can give only syntactically correct 
links. 

3. We specify details of each node using a window called a property window. 
When we specify details of a processing node, we should select the most 
appropriate program template from a list. 

Fig. 7 shows a Web transition diagram editor of T-Web system. 

A Web application generator. A Web application generator is a part of T- 
Web system, which gets intermediate documents a Web transition diagram 
editor generates, and generates source codes of server side programs and 
Web page templates. Our prototype system shown in Fig. 5 generates JSP 
documents written in HTML and servlet source codes written in Java. 
Generators have general-purpose program templates which are independent 
of specific domains of applications so that we can widely reuse them. The 
above prototype has 16 program templates which are mainly for database 
manipulations, session management and sending electronic mails. Fig. 8 
shows an example of program templates in T-Web system. In our templates, 
a word between special characters ’/*’ and ’*/’ represents a parameter. Spe- 
cial characters ’/**’ and ’**/’ mean a repetition part. As the generator 
rewrites above parameters to corresponding values, the program template 
becomes a complete program. 

To keep the standard level of security, all generated programs have additional 
codes as follows. 
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Fig. 7. A Web transition diagram editor of T-Web system 



1. After activated by a user’s request, each program checks whether user’s 
session id is valid. 

2. Then each program checks whether input data are correct according to 
specifications we give. 

3. After processing of business logic, each program checks whether output 
data are correct according to database specifications. Especially when the 
program updates database tables, it checks consistency between output 
data and specifications of the database table. 

If there is at least one error, the user’s request is redirected to an error page. 



4 Comparison of Two Approaches 

We compare annotation approach and diagram approach in terms of target ap- 
plications, developing processes and target users. We discuss target applications 
dividing into three viewpoints: application domains, flexibility and scalability. 
We discuss them according to the following criteria. 

1. Backgrounds show why the viewpoint is important to select a generator. 

2. Characteristics show the advantage and the disadvantage of two ap- 
proaches and compare them. 

3. Examples show our practical experience and other points to notice. 

4.1 Application Domains 

Backgrounds. Web applications can be generally classified into two types ac- 
cording to their functions: data-centric Web applications and control-centric 
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public class /*CLASSNAME*/ extends HttpServlet { 
public void doGet (HttpServletRequest request, 

HttpServletResponse response) 

throws ServletException, IOException { 
String myHttpsURL = "/*HTTPSBASEURI*/"+"/*PROGRAMURI*/" 

+"/*PACKAGE . */ "+" /*CLASSNAME*/ " ; 

String dbName = "/*DBNAME*/" ; 

String tbName = "/*TABLE1*/" ; 

String [] dbColNames = {/*"FIELDNAME1"*/}; 

/ **String PARAMETER = "";**/ 

Connection con = TWeb . doConnect (dbName) ; 

omit following parts 

} 

} 



Fig. 8. An example of program templates 



Web applications. Data-centric Web applications have side effect compu- 
tations such as operations to update databases, while control-centric Web 
applications concentrate on execution of complex business logic. 

Characteristics. Both our annotation approach and diagram approach con- 
centrate on data-centric Web applications. Our Web application generators 
aim to encapsulate complex tasks unique to Web applications such as session 
management and security checks, which are greater problems in data-centric 
Web applications than in control-centric ones. In general, both approaches 
can deal with applications having functions as follows: database manipu- 
lations, invocation of external programs and sending/receiving electronic 
mails. On the other hand, both approaches are not good at dealing with 
complex page transitions which depend on the results of programs. 
Annotation approach is useful to generate Web applications whose structure 
of Web page templates is complex, because we can use general Web page 
composers and authoring tools to compose flexible page layouts before the 
generation. Diagram approach is not good at developing such applications, 
especially clickable maps and multiple frames, because it is complicated work 
to connect data-flow relationships to the above components after the gener- 
ation. On the other hand, diagram approach doesn’t care a ratio of static 
parts and dynamic parts in each Web page, while annotation approach is not 
good at developing Web applications whose ratio of dynamic parts is high. 

Examples. We are successful in developing typical data-centric Web applica- 
tions using A- Web system and T-Web system respectively: shopping cart 
systems, guestbook systems, glossary systems, schedule organizing systems, 
member registration systems and reservation systems. The most appropri- 
ate generator mainly depends on the complexity of appearance of Web pages 
but not on application domains. When we want to develop more complicated 
Web applications, general software techniques such as object-oriented anal- 
ysis/design and the extension of them may be helpful [11]. 
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4.2 Flexibility 

Backgrounds. When we need new business logic to generate an intended Web 
application, it is a big problem how to add new program templates and com- 
ponents into generators. It is also important what types of architectures it 
can deal with, because we often have constraints on the execution environ- 
ment of the application. 

Characteristics. In annotation approach, we can set new program components 
and give annotations to invoke the program. In diagram approach, we have 
only to add new program templates, program components and documents of 
their specifications, because the generators usually load specifications of all 
program templates and components automatically when it starts up. Dia- 
gram approach seems to be more flexible than annotation approach, because 
program templates encapsulate invocation of external programs and it is 
easier to add new program templates. When we want to generate Web ap- 
plications based on various architectures, we can replace a part of generators 
in both approaches. Especially in diagram approach, we may have only to 
replace program templates to achieve the above goal. 

Examples. Using our generators, typical data-centric Web applications are gen- 
erated without new program templates and components. However, we pos- 
sibly need new business logic in the case of a complex shopping Web site. In 
that case, it requires high programming skills and much cost to test them 
to produce new program templates and components. We have implemented 
generators and made sure that our approaches can deal with CGI, ASP and 
JSP/servlet architectures. When we extend a generator so that it can deal 
with new architecture, it requires very high programming skills. In that case, 
diagram approach may be easier than annotation approach. 



4.3 Scalability 

Backgrounds. The scale of the intended application is also an important prob- 
lem as well as the complexity of Web page templates. 

Characteristics. In annotation approach, we can basically concentrate on tran- 
sitions between two Web pages, because the generator automatically gives 
a method to pass values of parameters from the origin page to the pages 
where they are used. Thus, even if the scale of an intended Web application 
becomes larger, the task of generation doesn’t become so complicated. In 
diagram approach, however, the larger the scale of an intended Web appli- 
cation becomes, the more complex the management of diagrams becomes. 
While we can make diagrams nested, it is still a big program how to manage 
specifications of databases which are used in several diagrams. 

Examples. Generators based on diagram approach can deal with no more than 
15 Web page templates easily. While 15 Web page templates are enough 
for typical Web applications, a complex shopping Web site may exceed the 
number of templates. It may be helpful to use site diagrams which represent 
the structure of the Web site and static relations among Web page templates. 
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4.4 Developing Processes 

Backgrounds. Web applications are characterized by three major design di- 
mensions: structure , navigation and presentation[ 10]. We discuss effective 
developing processes in each approach taking notice of the order of these 
three design activities. 

Characteristics. In annotation approach, we generally take the following de- 
veloping process. 

1. We first describe requirement specifications and analyze them. 

2. We decide what Web pages are needed in the application, and design data 
structure (as a structure model) which the application deals with. From 
the above two models, we design Web page templates (as a presenta- 
tion model) and implement them. We also design data-flow relationships 
among the application (as a navigation model) based on the above two 
models. 

3. From the implementation of Web page templates and the navigation 
model, we generate server side programs using the generator. 

In diagram approach, we generally take the following developing process. 

1. We first describe requirement specifications and analyze them. 

2. We decide what Web pages are needed, design data structure (as a struc- 
ture model) and design data-flow relationships among the application (as 
a navigation model) . From the above three models, we compose diagrams 
such as Web transition diagrams. 

3. From the diagrams, we generate server side programs and prototypes 
of Web page templates using the generator. And then, we design Web 
pages (as a presentation model) and revise generated templates. 

If we want to revise the appearance of Web pages after the generation easily, 
we should use XSLT and CSS to compose Web page templates in both 
approaches. 

Examples. Annotation approach has the advantage of composing prototypes 
of Web page templates early in the process and reusing them for the final 
product. We can take an iterative and incremental process composing Web 
page templates. On the other hand, it is the disadvantage that it’s hard 
to implement navigations as a prototype before composing Web page tem- 
plates. Diagram approach has the advantage of designing and implementing 
structure and navigation iteratively and incrementally. On the other hand, 
it is the disadvantage that we cannot start implementation of Web page 
templates before the generation. 



4.5 Target Users 

Backgrounds. The knowledge a generator requires is important for developers 
to use the generator. 

Characteristics. In annotation approach, Web page designers compose page 
templates before the generation. Users of the generator don’t need knowl- 
edge of markup languages, because an annotation editor points out where 
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and what types of annotations we can give. The users also don’t need knowl- 
edge of architecture of Web applications. On the other hand, the annotation 
based generator requires knowledge of data types, regular expression and the 
concept of session management. To develop advanced Web applications, it 
may require knowledge of APIs to invoke external programs. In diagram ap- 
proach, users don’t need knowledge of markup languages and programming 
languages, because program templates and components encapsulate them. 
The generator requires knowledge of Web interfaces, the concept of session 
management and basic architecture of Web applications. It also requires a 
skill in selecting and using general-purpose programs. 

Examples. Annotation approach is effective for programmers who have basic 
knowledge of software development. Diagram approach may be easier for 
inexperienced programmers, because most typical data-centric Web applica- 
tions have database manipulations and input checks, which are encapsulated 
by the generator. 

5 Related Work 

Currently there are many languages and tools to support development of Web 
applications. One of the widely used server side technology is so-called server 
side scripting such as ASP, JSP and PHP. We give fragments of program codes 
into Web page documents to describe dynamic parts of the Web page. However, 
we still have to give processing steps of session management and security checks 
in detail, because most of the program codes are procedural. Similarly, most 
support tools to generate a part of the above program codes require procedu- 
ral programming. Thus, the encapsulation of such techniques is not enough for 
inexperienced programmers. 

We may observe that there are many types of diagrams used for design or 
construction of Web applications. The extension of UML diagrams is one possible 
approach [11]. These diagrams are available to describe not only data-centric 
Web applications but also control-centric Web applications. However, it is not 
easy to use such diagrams for automatic construction because the descriptions 
are too detailed for inexperienced programmers to understand them. The aims 
are different in such techniques and our software generators. 

One of the systematic construction methods of Web applications is object- 
oriented hyper media design method [12]. This method is to produce interfaces 
of Web applications from abstract data types using three types of diagrams. It 
presents the developing process from requirements to the products and diagrams 
to use. While it is useful under specific conditions, it is not so flexible because 
the starting point of the developing process is always abstract data types. 

6 Conclusion 

We have presented two approaches to automatic construction of consistent and 
secure Web applications. We designed and implemented Web application gen- 
erators based on each approach. In this paper, we showed the roles of these 
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generators, and compared two approaches in terms of target applications, de- 
veloping processes and target users. Both approaches are powerful and flexible 
enough to construct typical data-centric Web applications efficiently. Especially 
annotation approach has the advantage of developing applications whose Web 
page templates have flexible page layouts. On the other hand, diagram approach 
has the advantage of rapid and iterative/incremental development. 

As our future work, we may try another approach which is based on the 
combination of two approaches. In this approach, we can develop programs and 
Web page templates concurrently. Such generator makes developing processes 
more flexible. 
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Abstract. Building web applications for mobile and other non-desktop devices 
using established methods often requires a tremendous development effort. One 
of the major challenges is to find sound software engineering approaches ena- 
bling the cost efficient application development for multiple devices of varying 
technical characteristics. A new approach is to single author web content in a 
device independent markup language, which gets then adapted to meet the spe- 
cial characteristics of the accessing device. This paper describes our approach 
to single authoring, which was developed in the course a large European re- 
search project. The project has developed a device-independent language pro- 
file based on XHTML 2.0 and implemented a compliant rendering engine. We 
focus on layout and pagination capabilities of the RIML (Renderer Independent 
Markup Language) and show how authors can be assisted by development tools 
supporting device independent authoring. 



1 Introduction 

The World Wide Web has established itself as one of the most important sources for 
information as well as a perfect infrastructure for applications, which need to be ac- 
cessible from anywhere. Web-enabled devices potentially offer access to globally 
adopted infrastructure. But why is this offer just potentially? Currently, most web 
content is optimised for the usage on a PC. This is still true despite the efforts of the 
Web Accessibility Initiative [1] which set standards for universal accessibility of web 
content. As more and more non-desktop devices enter the market, a convenient way 
to access web content and web applications using such devices is required. The in- 
dustry addressed that requirement for a limited set of applications particularly in the 
B2E (Business-to-Employee) domain by developing device specific versions of such 
applications. This application-specific approach tends to be too costly and not man- 
ageable if scaled to a large number of diverse devices and applications. Therefore, 
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more generic ways to prepare web content in a device-independent way are neces- 
sary. 

Various approaches have been proposed to address the challenge of high quality ap- 
plications at minimal costs. Some approaches (e.g. [2, 3]) automatically compile web 
content to fit on a target device. These approaches are based on HTML and use heu- 
ristics in addition to tag information to extract structure information, due to the fact 
that HTML is lacking the necessary semantics which is needed to perform the conver- 
sion to other markup languages. Therefore, some approaches replace HTML by an- 
other markup language which is semantically rich enough to serve as a basis for con- 
version [4, 5, 6], However, none of these proposals is standards-based and they were 
therefore not widely adopted in the marketplace. To open the path for a widely 
adopted device-independent markup language for authoring web content, two consid- 
erable efforts have been launched recently: a) The W3C chartered the Device Inde- 
pendence Group to establish specifications supporting single authoring, b) a consor- 
tium of six European companies, named CONSENSUS 1 [7], has built a prototype 
which implements authoring and conversion tools for a Renderer-lndependent 
Markup Language (RIML), which was also developed by this consortium. Both ef- 
forts cooperate intensively. The ultimate vision of device independence as stated in 
the charter of the W3C Device Independence working group [8] is to provide “Access 
to a Unified Web from Any Device in Any Context by Anyone”. In this paper we focus on 
layout and pagination capabilities of the RIML (Renderer Independent Markup Lan- 
guage) and show how authors can be assisted by development tools supporting device 
independent authoring. 



2 Requirements for Device Independency 

In this section we briefly cover the most important requirements for a device inde- 
pendent markup language and explain how the Consensus project has dealt with those 
in the RIML. Following the author once, display everywhere approach, an application 
is written once and gets then adapted to a particular device, which is accessing it. 

A perfect solution according to [9] should offer: 

a device independent markup language preserving the intent of the author, 
an adaptation system transforming it into a device specific form, 
means to retain the authors control over the final result/presentation. 

Concentrating on the developer’s part item 1 and 3 are the most important ones. To 
keep the intent of the author the markup language should offer semantically rich 



1 IST-Programme / KA4 / AL: IST-2001-4.3.2. The project CONSENSUS is supported by the 
European Community. This document does not represent the opinion of the European Com- 
munity. It is also the sole responsibility of the author and not the responsibility of the Euro- 
pean Community using any data that might appear therein. 
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markup elements. While XHTML 2.0 [10] provides certain means for device inde- 
pendent markup some necessary ingredients are missing. The RIML defines a lan- 
guage profile which is based on XHTML 2.0 and includes new elements for function- 
ality, which was missing in current standards or proposals. In the reminder of this 
section we give a brief overview of these. 

Due to the fact that devices widely vary in the amount of content their screens can 
accommodate there is a need for providing hints for content splitting and navigation 
between split up pieces of content. RIML therefore uses the section element of 
XHTML 2.0 as implicit hint for splitting and introduces new elements for combining 
layout with pagination and controlling the generation of navigation links between 
pages of split documents. 

To provide means for the development of interactive web applications (business ap- 
plications usually belong into this group) there is furthermore a need for elements 
dealing with form interaction. The XForms 1.0 [11] specification perfectly fulfills this 
requirement. The RIML language profile includes XForms 1.0, which strictly sepa- 
rates data from its presentation and keeps user interface elements device independ- 
ent 2 . 

Apart from the fact that a device independent language allows for the generic de- 
scription of a user interface for web applications, there should be means allowing for 
the inclusion of optional and alternative content. Even if this is somehow contradict- 
ing the author once approach, cause it allows for special versions of content for dif- 
ferent devices, it enables the author to retain the control over the result of the adapta- 
tion process and allows him to consider the specialities of particular device classes 
(e. g. the support for certain audio or video formats). The RIML language profile 
therefore includes SMIL [12] Basic content control plus some extensions. Content 
control furthermore allows for the selection of device (class) specific style sheets. 
Style attributes like font sizes and weights tend to be very device specific, therefore 
RIML authors should provide different style sheets for different target markup lan- 
guages and use the RIML content control mechanisms to select between them. 

An optimal layout of a page or presentation unit on a certain device (class) heavily 
depends on the special characteristics of it. A generic layout specification, which then 
is adapted to the device accessing the content, is hardly to achieve. The layout should 
facilitate the usability of the particular application and therefore utilize the special 
characteristics of the device, e. g. the available screen size. Therefore RIML supports 
the authoring of different layouts for different device classes in a RIML document. 
Using content control the author is able to define when to use which layout. 



2 The latest draft of the XHTML 2.0 specification now also includes an XForms 1.0 module. 
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In this paper we focus on layout and pagination capabilities of the RIML (Renderer 
Independent Markup Language) and show how authors can be assisted by develop- 
ment tools supporting this novel features. 



3 Layout and Pagination 

Layout refers to the arranging pieces of content on a presentation unit PU 3 . The way 
layout is specified is a crucial problem when developing device-independent applica- 
tions. Usually authors accomplish the layout by using frames and/or abusing tables 
for this purpose. Especially the latter is in conflict with the initial meaning of tables to 
serve as a structuring element, assembling multiple data records, each of which takes 
up exactly one row of the table. A generic layout specification, which then is adapted 
to the device accessing the content, is hardly to achieve. Automated layout genera- 
tion, as described in [13, 14] might help for certain use cases, like assembling the 
layout of a remote control UI on several devices, but removes the author’s control 
over the final result to a wide extent. 

The layout should facilitate the usability of the particular application and therefore 
utilize the special characteristics of the device, e.g. the available screen size. Consid- 
ering the limitation of the available screen size on certain devices another challenge 
of device independent authoring becomes apparent. Adapting to small screens re- 
quires the pagination (decomposition) of content, which in turn has to be taken into 
account, when specifying means for the layout of an authoring unit 4 . Usually some 
layout regions should be visible on all pages, e.g. menus and status bars; others in- 
clude content, which should get split into a sequence of presentation units the pagina- 
tion process generates. 

Automated pagination support was a main design goal for RIML. Other approaches 
assume selectors that explicitly define device dependent breaks (like [15, 16]), in 
effect, falling back to device related authoring. In contrast, a RIML author requires a 
minimal knowledge of how a desired layout will be paginated by the RIML adapta- 
tion system. In this respect, RIML is related to other approaches [17, 18]. It differs 
from these in that it supports generic HTML like (row, column) constructs for adap- 
tation, respectively applies adaptation to arbitrarily nested row and column structures. 



3 A presentation (PU) unit is this result of splitting the resource into a number of smaller units 
(PUs), which are presented to the user in a manner that is appropriate to the device (see 
DIWG2002]). 

4 An authoring unit is a piece of content the author is working with, but which is less than a 
document (see DIWG2002] ). 
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Layout in RIML 

Because of the reasons discussed in the last section RIML supports the specification 
of device (class) dependent layouts. Using Content Control the author is able to de- 
fine when to use which layout. The overall layout of a RIML document is defined by 
using elements specified in the RIML layout module. The layout module defines a set 
of container types: rows, columns, grids, as well as frames. Whereas containers define 
the overall structure of a layout definition, frames are used to fill the several regions 
of the layout with content. Using different layouts and Content Control for layout 
selection allows content to be organized differently, depending on the target device 
and its unique properties. If a frame cannot simultaneously accommodate all of the 
content assigned to it in the target language, the content needs to be paginated. Pagi- 
nation, the division of a RIML document into multiple pages, and navigation, the 
hyper linking among pages generated from a single authoring unit, are carried out 
automatically by the adaptation system. 

RIML defines the following layout containers: 

grid: A grid is a layout container allowing for the arrangement of container items in 
a grid layout. Container items can be (nested) other layout containers as well as 
frames. The parameter set of the grid element supports the specification of the maxi- 
mum permissible number of columns in a grid and the direction into which grid items 
will be led out (either horizontally or vertically). 

column: The column container is a special case of the grid container limiting the 
number of columns to one by definition. 

row: The row container is a special case of the grid container forcing all contained 
items to be laid out in a single row. 

frame: A frame is the only layout element content is assigned to, therefore a frame 
cannot contain any other layout elements. Each layout definition includes one or 
several frames. The assignment of content to a frame is done using the 
riml : frame Id element. 

Multiple frames can be used for the same presentation unit. Arranging frames differ- 
ently in containers produces different layouts. RIML’s Content Control gives the 
author a possibility for defining rules when to use which layout. While layout con- 
tainers arrange the layout of frames in different ways, the actual content is assigned to 
frames only. All content in the body part of the document must be included into sec- 
tions. Each section is assigned to a frame. The content of the section will be rendered 
within that frame (see Fig. 1). The content of a section will be ignored, whenever a 
section is not assigned to a frame or the frame does not exist. A RIML document 
should therefore define at least one frame in the document header. The overall layout 
of a document is defined by grouping multiple frames inside containers. RIML offers 
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different container types with differing layout and paginating behavior. All frames 
must be placed inside a container. The nesting of containers is allowed, i.e, a con- 
tainer can include other containers, whereas the innermost container must always 
include a frame. Therefore, a frame is the only type of container, which can be asso- 
ciated with content. 



<head> 

<riml : layout eccdc : deviceClassOneOf = "DeviceClass2 "> 
<riml:column riml : id= "root-col " > 

<riml: frame riml : id= "head" /> 

<riml: frame riml : id- " nav-menu " /> 

<riml: frame riml : id= "home " /> 

</riml : column> 

</riml : layout> 

</head> 

<body> 

<section id= "head-sec" riml : frameld= "head" > 
</section> 

<section id="menu-lsec" riml : frameld= "nav-menu"> 
</section> 

</body> 



Fig. 1. Assigning content to Frames 



Pagination in RIML 

XHTML 2.0 defines the block level element section as a means for structuring 
content. Apart from this a section defines an implicit page break, which is more 
“natural” than explicit page break markers. Therefore we decided to use sections as 
semantic hints for the adaptation system, when applying pagination. The author is 
therefore required to put all content, which should go onto the same screen (e. g. an 
input field and its label or a whole address form) into a section. Sections might be 
nested 5 , whereas the innermost sections never get split. As every section moves into a 
certain frame defined by the layout in the head part of the RIML document, it is often 
the case, that certain regions of the layout should not get distributed over several 
pages, because they contain information, which should appear on every page, even if 
the document gets split. Taking Fig. 2 as an example, the LogoFrame as well as 
FrameB are defined as non-paginating frames (see also Fig. 3). For Frame A pagina- 
tion was allowed (paginate is set to true). 



5 The current reference implementation does not support nested sections. 
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Fig. 2. Pagination and Layout Example 

Therefore the content assigned to the LogoFrame and FrameB remains over the se- 
quence of pages produced during adaptation. For FrameA the content gets distributed 
across all the pages the adaptation process produces. Fig. 3 shows a snippet of the 
markup, which was used to produce the results shown in Fig. 2. Section slO contains 
special markup for guiding the automated generation of links between split pages. 
According to the definition used here, navigation elements should be generated for 
every page, showing a link to the previous and next page respectively. Navigation 
links are generated if and only if pagination occurs. The navigation-links 
element offers attributes allowing the author to define the type of link to be produced 
and to which Frame it is related. In Fig. 3 the scope attribute refers to FrameA, there- 
fore navigation links are generated referring to pages containing sections from 
FrameA. 

The size of a frame is determined by the adaptation system in order to accomplish 
pagination. However, the author should be able to provide the system with meta- 
information regarding the intended width of a frame. The pagination process consid- 
ers the width of the virtual area on which the content gets finally rendered only (ren- 
dering surface), because this value can be reasonably defined. In case the browser 
supports no scrolling this value typically equals the physical screen width. Using the 
aforementioned frame elements the rendering surface can be divided into multiple 
parts. A frame serves as a bounding box in which the content will be laid out that it is 
assigned to. Using the following frame attributes the author is able to control the 
adaptation: 

minWidth - the minimum width in pixels - the frame should never be smaller than 
this specified width. 

pref erredWidth - the recommended width of a frame is usually specified as a 
percentage in relation to the actual width of the rendering surface of the device. Ab- 
solute pixel values for preferredWidth are also supported. 
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<head> 

<title>Pagination and Layout</title> 

<riml : layout> 

<riml: column riml : id= " coll " > 

<riml : frame r iml : id= " LogoFrame " riml :paginate=" false" 
riml : minWidth= "200" 
riml :pref erredWidth= " 400 " /> 

criml : frame riml : id="FrameA" riml :paginate=" true" 
riml : minWidth= "200" 
riml :pref erredWidth= " 400 " /> 

< r iml : frame r iml : id= " FrameB " riml :paginate=" false" 
riml : minWidth= "200" 
riml :pref erredWidth= " 400 " /> 

</riml : column> 

</riml : layout > 

</head> 

<body> 

<section id="sl" riml : frameld=" FrameA" > 

</section> 

<section id="s2" riml : frameId="FrameA"> 

</section> 

<section id="sl0" riml : frameld=" FrameB "> 
criml :navigation> 

criml : navigation- links riml : scope= " FrameA" 

riml : links= "previous " riml : linksValue=" relative-order" /> 
criml : navigation- links riml : scope=" FrameA" 

riml : links="next " riml : linksValue=" relative-order" /> 
c/riml :navigation> 
c/section> 

csection id="sll" riml : frameld=" LogoFrame "> 

cstrong>This is the sticky logo framec/strong> 
c/section> 
c/body> 



Fig. 3. Example Markup for Layout and Pagination 

The pref erredwidth attribute of a frame provides a hint to the adaptation process 
to determine the actual width of a frame. The author has to ensure that minWidth 
attributes in a RIML document layout can be obeyed for all devices. Therefore, the 
author should consider the device offering the smallest width of all targeted devices 
or better specify multiple layout definitions for different devices respectively device 
classes using Content Control. 

Given those hints, a pagination algorithm has enough knowledge to paginate without 
exceeding the screen surface width. A similar approach cannot be applied with re- 
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spect to container height, due to mentioned undetectable user preferences. The deter- 
mination of optimal height is based on two observations. 

First, we expect that most browsers support vertical scrolling. Vertical scrolling was 
shown to be acceptable from a usability point of view [19], in contrast to two- 
dimensional panning. Vertical scrolling was shown to be disturbing, if certain limits 
are exceeded [20]. To avoid the latter, we enable the user to reduce (resp. increase) 
the size limit which is applied during pagination. In support of this, the adaptation 
system is to insert corresponding control hyperlinks. In effect, the user exploits the 
visible outcome of pagination to avoid undue vertical scrolling depths. 

Besides Frames, the RIML furthermore defines other types of paginating elements, 
which are: paginatingGrid, paginatingRow and PaginatingColumn. With respect to 
pagination all of these container types operate on their child nodes, which might be 
either other containers or frames and which might be either paginating or not. To 
preserve a useable result, the pagination of nested layout elements is allowed for one 
branch only. Assuming that the hierarchy of layout elements forms a tree, this means 
that two paginating elements must not have a common ancestor. 

For non-interactive content i.e. printed pages, the content will be split when the page 
is full and page numbers are inserted to provide a basic means of navigation. Specifi- 
cations such as XML Print [17], CSS3 Paged Media [13], and XSL-FO [12] address 
that. 



4 Tool Support 

One major issue with new language proposals like RIML addressing Device Inde- 
pendence is the learning curve they require. The necessary authoring environment is 
often neglected in similar projects. Therefore, we decided to put considerable effort 
into the development of an authoring environment containing innovative tools, ena- 
bling the application developer to easily author in a device-independent way. Based 
on the Eclipse [21] open source platform, this authoring environment has been devel- 
oped as a plugin, gathering a set of views and editors, synchronised around a common 
document model. The Consensus authoring environment provides a whole set of 
tools. 

Besides an XML Editor consisting of a source text editor, code completion based on 
the RIML schemas, RIML language validation, and an XML tree editor, the toolset 
also provides tools, helping the developer to cope with the new concepts RIML intro- 
duce in a visual way. A Frames Layout View allows the author to get a first impres- 
sion how the document looks like, based oh the abstract layout he defined in the 
document. The concept of this view is to show an early version of the frames layout 
of the current RIML document. Depending on the device class one chooses, the view 
will show the frames layout, including in each frame the names (or ids) of the section 
that belong to this frame. The XML text in the text editor is highlighted depending on 
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how the author places the focus in the Frames Layout View. Fig. 4 and 5 show the 
Frames Layout View for a PC and a smart phone, respectively. 




Fig. 4. Frames Layout View for Device Class 4 




Fig. 5. Frames Layout View for Device Class 2 

RIML authors often need to know how the developed document will be paginated 
depending on a device class. Therefore, the RIML Device Dependent View provides 
an overview of how the document is paginated, how many pages are created, and 
what they contain. The view was implemented similar to an XML tree view, present- 
ing the split pages as a set of nodes. Fig. 6 shows an example of the device dependent 
view. 
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Fig. 6. Device Dependent View for Device Class 1 

Apart from these rather abstract views, we have also integrated a set of available 
device emulators, allowing the author to see the actual result of the adaptation process 
on a particular device. 



5 Conclusions 

The paper has discussed the two particular aspects layout and pagination of content 
authoring for non-desktop devices and respective solutions developed in the Consen- 
sus research project. Rather than just specifying technology and markup to support 
single authoring, the Consensus project undertook the effort to implement a reference 
implementation and a set of tools supporting the author in applying these new con- 
cepts. Our experience with the Consensus prototype proved the feasibility of the con- 
cepts developed by the project. Test applications are now being built and will undergo 
a field test under close supervision of usability experts, ensuring that the developed 
concepts and technology are not just feasible, but also meet usability requirements. In 
parallel to that, the project works in close cooperation with the W3C to standardize 
key concepts explored in the project. 
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Abstract. The growing demand for web applications and the new multi-user 
and multi-device requirements of these has led to the need for a structured and 
well-reasoned approach that helps both the application designer and the 
developer to produce a good quality product. On one hand we have the 
application designer, who has to describe all aspects of the application and 
manage the complexity of the tasks; on the other hand both the application 
customer and the developer need to validate and understand the designer's 
choices. To conciliate these needs, we propose a prototyping framework based 
on W2000 [ 1 ] [2] methodology: in this way the designer has a powerful tool to 
keep control over the web application, the customer has a point of reference for 
evaluating the design, and the developer can better understand the design thanks 
to the prototyped application. 



1 Introduction and Background 

During the development and design of web applications (WAs) there are many critical 
factors to be considered; the designer has to manage the individual aspects and must 
be able to predict the inevitable interactions between them. In order to manage WAs 
fully, the production of a mock-up application that prototypes the most important 
aspects as quickly as possible, with minimum investment of time and resources, is 
desirable. Szekely stated that prototyping involves building a small scale version of a 
complicated system in order to acquire the critical knowledge required to build a 
complete system [3]. Considering the relation between requirement elicitation and the 
mock-up application [4], the first version of the prototype will rarely get the users 
excited but the developers should encourage the users to give feedback for the 
revision of the prototypes and let them know that their feedback will be considered in 
the redesign. It might seem like a large effort is being spent on something that will 
eventually be discarded but this upstream activity represents good investment as it 
will avoid costly problems downstream. In the past few years, web application 
engineering (and thus prototyping) has been synonymous with ad hoc development 
and a lack of any structured methodological approach. Thus, several methodologies, 
supported by a suite of tools, have emerged in order to simplify and automate the 
development of WAs. HDM [5] proposes a model for hypermedia application design, 
which divides the conceptual schema into two categories: structural and navigational. 
Starting from the HDM model, Autoweb [6] introduces a variant notation - HDM-Lite 
- that adds a presentation schema to design a WA and store the schema together with 
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the data in a database. Jweb [7] provides a design/prototyping environment, 
integrating XML technology with HDM. In order to stay modular and flexible, one of 
the relevant features of the JWeb environment is the use of XML to the exchange of 
information among the different tools. RMM [8] methodology, HERA [9], 
ARANEUS project [10] and the WebRatio generation tool [11] represent other 
interesting approaches in a model-driven prototyping. Besides these approaches, 
UML [12] has been extended to model web specific elements [13]; starting from the 
use of UML as a language and the new WA features, HDM methodology has evolved 
into a W2000 approach. W2000 has been defined in response to the transformation of 
Web-based hypermedia from read-only navigational “repositories” to complex 
applications that combine navigational and operations in a sophisticated way. This 
paper describes a completely new prototyping architecture based on the W2000 
methodology and the results of the UWA project. 

The W2000 experience gave rise to the UWA Consortium [14], which in turn 
started the UWA project, which began in January 2000 and was completed in 
December 2002. This defines a set of methodologies, notations, and tools to tackle the 
main problems foreseen in the design of WAs. In addition, the project addresses the 
need for standard design and documentation to improve the possibility for exchanges 
and interoperability by extending the Unified Modeling Language (UML) as a design 
notation and using XML for internal representation. The project output, interesting for 
the purpose of prototyping, is the unified software environment, based on Rational 
Rose [15], which integrates the tools specific to each modeling activity. 



2 WAPS: Web Application Prototyping System 

In accordance with the W2000 methodology and the UWA design support 
environment, we propose a prototyping tool, called WAPS, which is able to 
understand the design model and create a mock-up application, with the benefits 
described above. Rose helps to design the WA in graphic format, using standard UML 
notation, in accordance with W2000 methodology; in order to obtain a “machine 
understandable” description we use another rational rose add-in: “Unisys Rose XML 
Tools” produced by Unisys [16] that exports the UML diagram into a standard XMI 
[17] output. Before choosing the XMI as the input format for WAPS, we considered 
two other reading model methods such as to read the rose petal file in raw mode or to 
produce an add-in to export the model into an xml-like format. The XMI solution 
seems the more standard, and appears to be suitable for our purpose; thus WAPS uses 
as input the XMI model description produced using the Unisys add-in which exports 
the Rational Rose W2000 model designed with the UWA add-in. As mentioned 
above, W2000 methodology and the UWA design tool were thought to be more 
appropriate for the design of WAs than for prototyping purposes. Therefore it is 
necessary to complete the WA model in order to support the prototyping process. It is 
clear that the complementary information tool must be a Rational Rose add-in 
(WAPS-Add-in), which provides a uniform environment for designing (with the 
UWA add-in) and prototyping WAs. The prototyping information is exported at the 
same time as the design information, in XMI format by the Unisys add-in. 

The WAPS architecture is composed of several modules: for the most part they 
make up the run-time environment (WAPS-RT), the other part provides a set of tools 
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(the “lnstanceDB Manager” and all the “ Rational Rose Add-in” in Fig. 1) to get 
WAPS-RT ready, such as the tool to insert the application data. The run-time 
environment, the WAPS core, has the main task of creating a mock-up application 
starting from the W2000 model in XMI format. 

In accordance with the modular structure of the W2000 methodology and the 
various aspects of WA, it is possible to identify a clear n-tier architecture for the 
WAPS run-time environment. The architecture was born as a generalization of the 
three-layer software architecture used to develop a pilot application based on W2000 
methodology [18]. This choice was highly suitable for the W2000 methodology: just 
as the model examines one by one the different aspects (information, navigation, 
publishing) of WAs in order to manage their complexity, each architecture layer 
manages a single aspect (or a pool of aspects), and provides services to the other 
levels. Since WAPS is based on an XMI description, it’s clear that all the data 
managed in the modules are in XML format; furthermore, all interaction between 
modules must be in the same format; the XML verbosity and adaptability ensure good 
portability and scalability of the whole system; furthermore, after checking the 
environment, it is possible to distribute the modules on several machines using web 
service techniques based on XML messages. 




Fig. 1. WAPS architecture 



In the WAPS-RT design, each level and thus each module manages a particular 

WA aspect; in detail: 

• Schema Browser: WAPS uses the WA schema based on the W2000 model to 
prototype the WA. The schema is the XMI description exported from Rational 
Rose. This module allows a unique entry point to the WA schema, hiding the 
complexity in order to manage the XMI in raw mode. The module provides a set of 
schema APIs (S-API) to navigate the WA model via W2000 primitives. 

• Core Engine: This module corresponds to the business level for a three-tier 
application. This module has the task of understanding the requests from the Object 
Builder, using the S-API of the schema browser to compose the reply schema that 
will contain the application data taken from the Instance DB. Since this module 
creates the reply schema (and thus the page that the user will see), it is clear that all 





WAPS: Web Application Prototyping System 259 



design customizations take effect at this stage. Users not only access the 
information but interact through the operations. In the future, this module will 
provide standard methods to implement the operations. Obviously the operation 
will be expressed as manipulation of the W2000 model elements. Thus, the task of 
adding a product to the shopping bag is translated in WAPS as adding an instance 
of PRODUCT to the dynamic collection called “Shopping bag”. In this way it is 
possible to describe complex operations as a sequence of basic ones performed on 
model elements. The importance of this piece of the framework is plain, if we 
consider its function, and it is so big that we may call it the “Core Engine”. 

• Object Builder: this module is the door to WAPS systems: the user request comes 
in, the prototyped page goes out. The module moves the request to the Core Engine 
and receives the response in XML-like format. Its main task is to apply a template 
to make the page visible. In order to manage several devices, WAPS uses the XSLT 
transformation to obtain HTML or WML pages in XHTML [19] format. XHTML is 
the keystone in W3C's effort to create standards that provide richer web sites on an 
ever increasing range of browser platforms. Browsers are not limited to desktops 
and include mobile phones and PDAs. Additionally, Cascading Style Sheets (CSS) 
can be used to control the content presentation. The use of CSS to manage the 
multi-device task is quite suitable for the prototyping purpose where the layout 
aspects are to provide an idea of the final layout. 

WAPS uses several information repositories to store the data: the XMI schema 
described earlier, the Instance DB and the publishing model template. 

The Instance DB is an E-R database that contains the data that will be showed to 
the user. The E-R schema stores the data in meta-structures derived from the W2000 
model; thus the schema is fixed and doesn’t change with the domain of the WA being 
prototyped. The Publishing Model Template is a E-R database which contains the 
references to the visualization template to create the page. There are two kinds of 
template: the default templates, which are generic and will be used to view the model 
structures such as the publishing unit, and the custom template, which the designer 
can create and associate with a specific page or publishing unit. 

The Instance DB manager is a specific tool to populate the Instance-DB: it 
accesses the WA schema to understand the data structure in order to guide the 
operator through the schema to insert meaningful data. 



3 Conclusion and Future Work 

This paper has concentrated on the model-driven prototyping of web applications. We 
propose a prototyping system called WAPS that allows the designer to produce a WA 
mock-up starting from its W2000 model drawn up using Rational Rose, In accordance 
with the W2000 model structures, the multi-layer WAPS architecture encapsulates in 
each level a specific aspect; this, combined with the extensive use of XML-like 
format (to describe the model and the application data, and to exchange information 
between modules) ensures good scalability and adaptability of the environment. 

Future work is evolving in two directions: the first aims to offer a better support for 
operation and customization and to improve the quality of the prototype; the second 
aims to produce support tools: considering the importance of application data in 
obtaining a good prototype, we are thinking of creating a tool that populates the 
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Instance-DB with data from statistics algorithms. As for the quality of application 
data, we are studying the features offered by the semantic web in order to add a new 
semantic layer to WAPS. 
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Abstract. In recent years numerous Web application modeling lan- 
guages have been developed and others improved. There has, however, 
been little research on how these languages may be simulated. Simula- 
tion of models constructed using design languages allows early evaluation 
and prevents unnecessary Web code development and implementation. 
It can therefore significantly reduce the design cycle time and cost. This 
paper introduces a Web application simulation model framework that 
was designed to be compatible with existing modeling languages. This 
was accomplished by specifically identifying the objectives of a simu- 
lation language and contrasting this with those of design models. The 
simulation model supports analysis of simulations from four key Web ap- 
plication perspectives (and hence the model is constructed around these 
perspectives) namely: presentation, navigation, functionality and con- 
tent. We argue that with this approach substantial inferences about the 
quality of the design can be drawn from simulation of the Web applica- 
tion model. 



1 Introduction 

Web applications projects have reached a level of complexity that demand mod- 
eling techniques to tackle their design intricacies. Several modeling languages 
such as OOHDM [1], UML [2] and WebML [3] have been proposed as aids to 
Web application design, development, and implementation. Although these mod- 
eling languages provide an overview of the system, there is still the need for some 
coding to get a first glimpse of the interaction amongst the systems components. 

A desirable approach would be to simulate the design without any imple- 
mentation whatsoever. Simulation has been widely and successfully used in 
the hardware design field. In fact, Hardware Description Languages (HDLs) 
such as VHDL (Very High-Speed Integrated Circuit Hardware Description Lan- 
guage) [4,5], are considered an indispensable modeling tool. Simulation of VHDL 
models is performed without a physical implementation thereby reducing both 
cost and design time. But if modeling of Web applications has been thoroughly 
addressed, simulation of the models themselves is little researched. So far, sim- 
ulation has only been used to evaluate Web systems performance [6,7,8]. 
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We feel that Web modeling languages have reached a maturity level that 
permits the present attempt of defining a framework for Web application simu- 
lation. We aim to build the foundations of a simulation model based on a Web 
Description Language (WDL) , developed specifically to assist on the simulation 
process, permitting the assessment of Web application designs regardless of the 
modeling language chosen. 

The next section describes the WDL Simulation model, and in section 3 
we outline the objectives, input stimuli, results, and analysis of the simulation. 
Finally Section 4 makes a brief summary and draws conclusions. 



2 The WDL Simulation Model 

Simulation consists on the observation of a system’s response over time when a 
known set of stimuli is present at its inputs [9, 10]. In this paper, simulation is 
regarded as the evaluation of the extent to which the Web application design 
(represented using some modeling notation) supports interaction with the user, 
and aims to monitor and draw inferences of what occurs internally. 

To simulate a design a description of its components has to be available. To 
deal with the specificities of Web applications a description language (WDL) was 
developed which captures both the structural and dynamics of each component, 
similar to VHDL, where for each component a definition of the interface and 
functionalities is explicitly declared. The main purpose of WDL is not to model 
the Web application but to enable the simulation. 

For the designer the most important aspects of the application that need to be 
assessed are its graphical interface, the navigation network structure, the scripts 
interface and workflow, and data exchanged among the components. In fact, 
existing models such as the Dexter Hypertext Reference Model [11], HDM [12], 
and WebML [13,14], tend to decompose systems along these lines [15] empha- 
sizing the use of different layers to tackle distinct aspects of Web applications. 
For this reason, our simulation model focuses on four different perspectives cor- 
responding to four different layers, namely: presentation, navigation, functional, 
and content, as Fig.l shows. Also a User Interaction Model was considered to 
reflect the possible actions that may be undertaken by a user, restricting the 
simulation stimuli to a well known set. Each layer, being an orthogonal aspect 
of the design, enables analysis from a distinct point of view. Simulation on these 
four layers requires the ability of the modeling language to describe in detail 
components structure, workflow, and behavior, and, therefore, a set of require- 
ments must be met for a high degree of accuracy simulation. If, however, the 
modeling language used does not encompass all the requirements, it does not 
automatically preclude simulation but will imply a lower simulation detail. The 
set of requirements the modeling languages should meet are enumerated and 
described in [16]. 
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Fig. 1 . The WDL Simulation Model 



3 Layer Definition 

The Presentation Layer Simulation relies on its stimuli to evaluate the 
system under analysis. Since the system is a Web application, its change in state 
is essentially due to user inputs through the graphical interface - the Presentation 
layer. Our simulation model reflects this by wrapping the Presentation layer 
around the remaining preventing the user to directly interact with them. 

This layer deals with the description of the page’s look and feel, namely the 
user data input components, information components, and hyperlinks. However, 
it does not attempt to interpret the user actions; its semantics are dealt with on 
the remaining layers. As a consequence, user actions that potentially alter the 
state of the system are initiated on the Presentation layer but acquire significance 
and meaning on the remaining layers. Stimuli of this layer come mainly from the 
user interaction through interface components - concrete and clearly identified 
HTML elements with well-defined structure and behavior. 

Results from the simulation are essentially the rendering of the model’s 
graphical user interface description by displaying the active pages, and the in- 
teraction with the remaining layers - the navigational consequences of the user 
browsing, the functional script outcomes, and the data included and displayed. 
Assessment of the user interface is this layer’s simulation main purpose, namely 
the graphical realization of hyperlinks and data input elements. Furthermore, 
the interface may pose some interaction limitations, namely on a data input 
level, which must be properly assessed by the designer. 

The Navigation Layer On this layer, pages are considered as macro 
blocks or abstractions of the pages displayed by the Presentation layer - they 
are simply containers of navigation components. Simulation of the navigation 
path through the Web pages set due to the user interaction with the interface 
via the Presentation layer is performed. It is on this layer that some of the 
signals that trigger script processing have their origin. Simulation performed on 
this layer is driven by user interaction with the displayed components of the 
Web application. The user interacts with the pages on the Presentation layer 
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and the possible actions performed belong to a well-defined finite set - following 
hyperlinks, mouse actions or buttons selection. Stimuli that drive the simulation 
are a generalization of the inputs in stereotyped categories such as clicking on 
links or the action of submitting data to the server, and script actions that imply 
navigation changes. However, the specific data input contents from the user are 
not considered in this layer - that will be dealt with on the Functional layer. 
Throughout the simulation the designer may observe the different navigation 
paths the user has available to chose from due to changes in the active pages. 
Different windows are opened in response to diverging navigation trails and, 
consequently, several new links become available. The designer may trace which 
user actions led to the opening, activation or closing of a specific window, and 
the correspondingly set of navigation elements available. One of the goals of 
simulating this layer is the verification of the navigation flow and assertion of 
the navigation path due to user and script actions. 

The Functional Layer It is on this layer that the application acquires 
its dynamic aspect and where the client and server-side scripts are managed. 
Furthermore, this layer interacts with the Presentation layer for dynamic page 
construction, and with the Navigation layer for internal changes in the active 
pages. Data input in Web forms as well as any state information are the principal 
sources of information obtained from users. Scripts process this data from the 
Presentation layer and stored data from the Content layer, and are triggered 
by other scripts or by user action via the Navigation layer. Resulting from the 
simulation, data flow exchange between the script components and components 
on the remaining layers are observed. The data used by a specific script and its 
results are also displayed. What scripts are running at a particular point in time 
and their contribution to the overall Web application dynamics are another of the 
simulation results. Furthermore, the assessment of the process workflow assures 
that the Web application dynamics is in agreement with the design requirements. 

The Content Layer All data stored on the system is managed by this 
layer which is a main information resource of the Functional and Presentation 
layers. It manages all the data used by the Web application - whether from 
databases, files, or more volatile data such as Web session variables. The stimuli 
of the content layer simulation are the data write and read command set such 
as a database write operation or reading a client’s cookie ID. 

The information flow amongst the different layers and all the actions that 
involve data manipulation are displayed, enabling assessment of the Web appli- 
cation content management. 



4 Conclusion and Future Work 

A framework for the simulation of Web application design models is presented. 
By using a Web Description Language that maps the design into an enumeration 
of the structural and behavioral characteristics, the WDL Simulation model is 
able to evaluate the design from four orthogonal perspectives, namely: Presen- 
tation, Navigation, Functional, and Content, which relate to the four important 
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analysis a designer has to conduct to verify the Web application and validate its 
requirements. By simulating the application directly from the model, and with- 
out any code implementation, substantial project time reduction is achieved. 
A prototype is being developed to simulate designs using several modeling lan- 
guages such as UML or WebML. Furthermore, the next logical step is to use 
WDL to automatically synthesize the code. This, as it happens in the hardware 
design field, would provide designers with powerful tools for the development 
of complex Web applications. A fuller description of the layers and especially 
the interfaces between them is available in an extended version of this paper 
published as an Technical Report [16]. 
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Abstract. A sales advisory system is a tool supporting customers in the deci- 
sion-making and buying process by interactive and personalized requirements 
elicitation and the provision of comprehensible product proposals and expla- 
nations. The particular challenges when building such systems lie in the strong 
interdependencies between the recommendation and personalization logic and 
the corresponding adaptive, web-based user interface. The Advisor SUITE tool 
described in this paper is a system that follows a consistent knowledge-based 
approach for all tasks that are required to build such intelligent sales advisory 
systems for arbitrary domains. The development of advisory applications is 
based on a conceptual model of online sales dialogues, a “Model-View- 
Controller” application architecture, a generic controller component, as well as 
(semi-)automatic, template-based web page generation. Experiences from 
various real-world applications show that the knowledge-based approach and 
the corresponding graphical tools of Advisor Suite significantly accelerate the 
development and maintenance process for such applications. 



1 Introduction and Overview 

Online customers are increasingly overwhelmed by the variety of comparable 
products or services available on the online channel. Web-based sales assistance and 
recommendation systems are a means for supporting online customers in their product 
selection and decision-making processes. These systems provide the best value for the 
customers when they simulate the behavior of a real sales assistant. Therefore, they 
acquire the customer’s real needs in a personalized dialogue ([1], [2]), and come up 
with a suitable set of proposals and provide adequate explanations for these proposals, 
which is required to increase the customer’s confidence in his buying decision. 

The purpose of the ADVISOR SUITE framework described in this paper is to provide 
a domain-independent tool for integrated development and maintenance of such web- 
based sales advisory applications. At the core, ADVISOR SUITE is an expert system 
where the knowledge of the domain expert is made explicit in a declarative 
knowledge base. This knowledge both comprises the recommendation guidelines of 
the domain, as well as information about how the real customer requirements have to 
be elicited, i.e., knowledge about efficient sales dialogues. The main challenge of 
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such an approach is that there are strong interrelations between these two types of 
knowledge. Typically, the user’s preferences are acquired by asking questions in an 
interactive dialogue. The user’s answers (i.e., his/her profile) obviously influence the 
personalized set of products to be recommended, but also determine the further 
dialogue flow which should be adapted, e.g., to the user’s skill level. 

Consequently, the web pages used in the online dialogue must be extremely 
flexible and dynamic such that changes in the knowledge base are immediately taken 
into account and do not require manual adaptation of the dynamic HTML code. 
Nonetheless, the web pages have to be comprehensible and editable by a Web 
developer who aligns the pages’ style to the corporate layout or integrates the 
application into an existing web site. 



Knowledge Acquisition Tools 
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Fig. 1 . Architecture of the framework 




Virtual advisor session 



Fig. 1 shows the overall architecture of the ADVISOR SUITE framework. The required 
knowledge is acquired using graphical knowledge acquisition tools and stored in a 
common repository. The server component utilizes that knowledge and creates 
interaction and personalization agents that manage the user input for each advisory 
session. Before the system is started, the needed web pages for the dialogue and a 
generic controller component implementing the personalization logic are generated. 



2 Combining the Recommendation Logic and Adaptation Logic 

In our approach, the customer properties are the glue between the recommendation 
logic and the personalized adaptation of the user interface and the dialogue. First, 
these properties, i.e., the user’s interests and preferences, determine the products to be 
proposed and their degree of fit with the requirements. The possible values (answers) 
for the properties are typically finite and pre-defined. The computation of suitable 
products is based on a priority-based filtering technique similar to [3] and declarative 
rules like, 

“If the customer’s experience in the domain is low and he has limited financial 
resen’es, we propose low-risk investments. ' n 



1 Simplified example taken from the domain of online investment advisory. 






268 



D. Jannach and G. Kreutler 



Such advisory rules are modeled in a graphical tool and are expressed in a high- 
level, end-user oriented language; the individual ranking of the remaining products is 
based on a standard personalization technique [4] and the evaluation of the products’ 
utilities for the customer. More details on the knowledge-based recommendation 
approach can be found in [5]. 

On the other hand, the customer properties also steer the interaction process, i.e., 
the sequence of the dialogue pages. In many cases, for instance, the interaction style 
depends on the user’s self estimate of his knowledge level in the domain. Expert users 
can be asked fewer, but more complex and more technical questions, novice users 
might need more help and a different form of explanations. Within the ADVISOR 
SUITE framework, this adaptation and personalization knowledge that an experienced 
sales agent will have is also made explicit. Again, the personalization process is 
driven by the customer properties and made explicit with rules like, 

“If the user has limited knowledge on the domain, proceed to a page where he is 
asked if he wants to have a look on more introductory material. ” 




Fig. 2. Modeling the dialogue 

These rules are maintained with the help of a special development tool which is 
depicted in Fig. 2. In particular, the used modeling approach is based on a simple but 
general conceptual model of a sales advisory dialogue, where the major concepts are 
chosen in a form such that they are close to the resulting web application and use a 
non-technical representation: Basically, sales assistance dialogues consist of dialogue 
steps (pages) that contain one or more questions on customer preferences or desired 
product properties. Each dialogue step can have a number of possible successor 
pages, whereby the actual successor page is determined dynamically based on the 
personalization rules described above. A dialogue can have “special steps”, like hints 
on conflicting requirements or additional information or result and explanation pages. 
Our experiences from several practical applications show that domain experts are able 
to cope with this level of complexity and can describe the structure of a good dialogue 
using these concepts after a very short training phase. An important factor for user 
acceptance also lies in the conceptual integration of this knowledge with the 
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recommendation logic, e.g., the same customer properties used for product filtering 
now appear as questions in the dialogue; the language for expressing complex 
conditions is the same as for page successor relations and for filtering rules. 
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Fig. 3. Structure of a dialogue page 



3 User Interface Generation 

The dynamic HTML pages of the final application have to be very flexible and must 
immediately reflect changes in the knowledge base, e.g., when a new question is 
defined. Moreover, they have to be simple enough such that they can be easily 
adapted by a Web developer who for instance wants to change the layout of the pages. 
Advisor Suite uses the following basic techniques to deal with that problem. First, 
we make extensive use of Custom Tags [6] which are syntactically similar to standard 
HTML tags, but implement application-specific functionality. The usage of these tags 
helps us both to avoid the problematic mixture of static HTML-code with procedural 
scripting code and also leads to a more clear and legible page source code. On the 
other hand, we also rely on automatic web page generation based on an elaborated 
template mechanism. Each dialogue page is actually built up from different, 
predefined areas and like question or explanation areas, headers and footers. The GUI 
Generation Module in Fig. 1 automatically assembles and parameterizes the needed 
web pages from small, pre-defined templates such that the basic dialogue can be 
generated (in a rapid prototyping process) without programming. Fig. 3 shows the 
layout and the different areas of such a generated page. 



4 Conclusions 

Over the last years, several approaches (e.g., [7], [8], [9], [10], and [11]) have been 
presented that aim at applying state-of-the-art Software Engineering practices to the 
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specific requirements of the development of web applications. To some extent, our 
work can be seen as an implementation of best-practices from these approaches for a 
specific application domain: The design process is based on a generic, conceptual 
model of a sales advisory application; the separation between the business logic, 
controller, and presentation layers is very rigid. As such, our system shows the 
practical applicability of the basic idea of these approaches. 

However, our work differs from most of the above-mentioned approaches as we 
want to support the full development process up to the automatic generation of the 
web pages; although we limit ourselves to a specific class of web-based systems, a 
broader analysis of the applicability of the presented ideas will be part of our future 
work. Another difference lies in the choice of the modeling notation, where we 
deliberately did not use a standard technique, e.g., based on UML (Unified Modeling 
Language). With the goal of short training times for the domain expert, we opted for a 
proprietary, end-user oriented notation with a defined semantics that is needed for 
automated application generation. Our future work will include a detailed evaluation 
of the applicability of standard modeling techniques from the field of Software 
Engineering for domain experts with limited background in that area. 
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Abstract. Web sites are public representations of corporations, busi- 
nesses and governmental bodies. As such they require proper mainte- 
nance to ensure that accurate and updated information is presented ad- 
equately. Dynamically created Web sites sometimes are not an option: 
they add a computational overhead on the server and make the auto- 
matic indexing of pages difficult. Static Web sites may grow to an extent 
that manually performing maintenance operations becomes unfeasible: 
automatic or semi-automatic means ought to be put in place. In this 
paper we explain one such approach, using software agents to look af- 
ter large data-intensive Web sites. Our maintenance operations are per- 
formed by a team of autonomous agents that communicate with each 
other as well as with Web masters or other human agents. Our approach 
can be naturally incorporated into existing Web sites and its use can be 
gradually extended to encompass larger portions of the site. Because our 
agents are independent, their individual malfunctioning should not stop 
the maintenance effort as a whole. 



1 Introduction 

Web sites provide to the public at large representations of corporations, busi- 
nesses and governmental bodies. Increasingly Web sites provide the first (and 
sometimes only) form of contact between the general public and the actual or- 
ganisation. It is imperative that proper care and attention be placed into the 
design of the Web site: the look-and-feel of pages, the provision of maps for the 
site, the amount and positioning of information, and so on [1], However, an in- 
dependent and equally important (or even more important) issue concerns the 
actual contents of the Web site: how accurate and updated the information pre- 
sented is and what mechanisms are in place to support their maintenance. This 
is a particularly sensitive issue in data-intensive Web sites, where data suffers 
constant or frequent updates and there may be many sources of data involved. 

* Partially supported by the Brazilian Research Council (CNPq) grant no. 55.2197/02- 
5 (Project SiteFix - Adapting Web Sites to Perform Information Retrieval Tasks). 
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We propose an automatic approach to Web site management via a team 
of software agents: a collection of independent and communicating pieces of 
software will share the responsibility and computational effort of looking after 
a large data-intensive Web site, managing static Web pages. We introduce an 
abstract and generic architecture and describe how it can be implemented using 
existing technologies and standards. 

Dynamic-page techniques [2,3] have been used to tackle the contents maite- 
nance problem. However, static pages still have an important place as they are 
simpler to manage, require less computational power and are visible to search en- 
gines. It should be clear that we do not propose a replacement for dynamic pages: 
information which usually resides in a database or that needs some computation 
before presentation are clearly better addressed via dynamic generation. Our ap- 
proach targets pieces of information which do not require complex computations, 
changing only their values and, in many cases, not being stored in databases. 
Both approaches should co-exist and be used in a single Web site application. 

This paper is organised as follows. In the next section we address some of 
the issues related with using agents to carry out Web site maintenance. In Sec- 
tion 3 we explain our proposed architecture and its components. In Section 4 we 
present an example to illustrate our approach. Section 5 contrasts our approach 
with existing work and in Section 6 we draw conclusions and discuss the work 
presented. 

2 Web Site Maintenance with Agents 

The idea to employ autonomous agents to perform Web site maintenance comes 
from the fact that maintenance tasks are usually small, well defined and recur- 
rent. To our knowledge, no-one has attempted this before: simple and robust 
software agents can be designed to carry out stereotypical tasks and be reused 
and customised to suit particular needs. 

Typical maintenance tasks our agents can handle are those involving updat- 
ing a piece of information and publishing it on a Web page. In order to specify 
this sort of task, it is only necessary to specify the information item, its data 
source, the frequency of update and the page where it is published. As a re- 
sult, the amount of knowledge agents need is relatively small. Agents can be 
implemented as simple lightweight programs, using only the necessary system 
resources. The main advantages of using agents are [4,5]: 

— proactiveness - agents are proactive, i.e., they take action when necessary. 

— autonomy - each agent is autonomous, being able to perform its task on its 
own with little or no human intervention. 

— social ability - agents can send and receive messages to the Webmaster, 
making it easier to follow maintenance activities. 

It is important to note that the proposed approach is not a replacement for 
dynamic pages techniques, such as ASP [3] and JSP [2], Dynamic pages comprises 
a widely adopted solution for maintaining the information updated. Although 
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any piece of information that changes over time can be regarded as dynamic, we 
can make a distinction between two types of dynamic information: (1) pieces of 
information which result from a computation, often requiring parameters given 
by users or coming from other source. (2) pieces of information that only have 
their values changed periodically. Once instantiated, this sort of information can 
be presented in static Web pages, which are simpler to be served to users and 
easier to be found by search engines. 

Our approach can co-exist with dynamic pages in a Web site, since the main- 
tenance agents are designed only to maintain static Web pages. It is an alterna- 
tive for maintaining static pages which contains dynamic pieces of information 
of type (2) as explained earlier. Note that we have not yet addressed other dy- 
namic aspects that can appear in the navigation structure and presentation of a 
Web site application. Another important feature of our approach is the ability 
to update content and visualisation separately. That allows, for instance, chang- 
ing completely the look-and-feel of a Web page without affecting the content 
specification or its data source. As benefits, our approach keeps Web pages au- 
tomatically up-to-date, speeding up the maintenance process. Since it requires 
fewer technical personnel (Webmaster), it also helps to reduce the overall costs 
of maintaining a Web site. 



3 An Agent-Based Architecture for Web Site 
Maintenance 

In this section we describe the components of our architecture, their details, 
how they relate to each other and how we implemented them. It is worth point- 
ing out that the architecture herewith described could have been implemented 
rather differently, using distinct communication infrastructures, different pro- 
gramming languages and even different notations to specify our agents with. 
In Fig. 1 we show a diagrammatic ac- 
count of our architecture. The Web 
Master is shown on the right-hand side 
interacting with the Web pages (white 
arrow): he/she annotates the HTML 
files with specifications of agents to 
be started up by the Scanner Agent, 
shown as a grey circle. The Scanner 
Agent is responsible for going through a given directory where the HTML files 
are stored and scans these for the annotations specifying agents. For each anno- 
tation found in the HTML file, a corresponding agent (black circles) is started 
up the complete set of agents obtained at the end of the scanning process is 
called the Team of Agents looking after the Web site, updating the same Web 
pages that gave rise to them (vertical arrows in the diagram). The Web Master 
and the Team of Agents communicate via message-passing (black two-way arrow 
in the diagram). We explain below each component of the architecture. 




Fig. 1 . Agent-Based Architecture 
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3.1 Annotated Web Pages 

The Webmaster adds annotations to individual Web pages, that is, to particular 
HTML [6] files. Since these annotations become part of an HTML file which 
Web browsers need to display, they must be innocuous , that is, they should not 
alter the rendering of the HTML by any browser. We achieve this by employing 
the HTML comment tag “<! — ... — >” to wrap our annotations. 

The actual annotations that go within the HTML comment tags are a special- 
purpose XML [7] construct of the form 

<agent info=" Infold" type=" AgType" param="j4gPar’ams"></agent> 
where Infold is a label to uniquely identify the piece of information within the 
Web site, AgType is the type of agent to be started and AgParams is an optional 
attribute with any parameters which ought to be passed on to the agent being 
started up. 

The information identification Infold allows content agents (explained be- 
low) to refer to the particular portion of the page they should look after. A 
single agent, the publisher agent explained below, is responsible for updating 
the annotations of a Web page. The scanner agent assigns to each page with 
at least one annotation a publisher agent; the annotations themselves cause a 
number of content agents to be started up these should look after particular 
pieces of information on the page. Whenever there are changes to be carried 
out on the HTML file, they are done via the publisher agent. This arrangement 
solves any issues arising from concurrent updates of a file and preserves the 
separation of contents and presentation matters. We exploit a fully distributed 
coordination mechanism, explained below, by means of which content agents 
independently offer their updates which are published by the publisher agent. 
The information identifica- 
tion Infold labels a piece 
of information within a Web 
site, allowing us to take ad- 
vantage of one agent’s effort 
for more than one page. As 
an agent is started up for 
an annotation to look after 
a piece of information, we might need the same information elsewhere within 
the Web site. If the same information appears elsewhere in the site then the 
Webmaster should annotate it with the same label - the agent looking after 
the information will interact with the publisher agent of the pages where the 
information appears and that have been annotated. If, however, the Webmaster 
accidentally uses a new label for a piece of information already associated with 
an agent, a new agent will be started up and will look after the information. 
This will not affect the Web site as the agents will independently look after the 
same information, but there will duplication of effort. 

Our annotations also work as delimiters for the HTML portion that the 
agents should be looking after, that is, the part of the page they are responsi- 
ble for monitoring and/or updating. In order to tell agents where the portion 



<html><head><title>Weather Forecast. . . </title></head> 
<body> 

Current Temperature 

<! — <agent info="temp" type=" weather Ag" 
param=" [every, 15 , min] "> — > 

30 <! — </agent> — > °C 
</body></html> 



Fig. 2. Sample Annotation for Web Pages 
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they should be looking after ends, we split the “<agent . . . >” and “</agent>” 
tags, (using HTML comments around them) and enclosing the portion of HTML 
within the two. We show in Fig. 2 a complete example of an annotation. In it, a 
specific item of information of an HTML file is enclosed within tags <agent . . . > 
and </agent>. We have associated the agent weatherAg with this portion of the 
HTML file which will be responsible for updating the information every 15 min- 
utes. The actual kinds of agents and their parameters are extensible. Webmasters 
may create annotations and associate them to special purpose agents which they 
also develop. We have explored a class of such agents which we explain below. 

The Scanner Agent - The scanning process is itself carried out by an agent. 
This agent is constantly running, checking for new annotations in the files 
or changes to existing annotations. When a new annotation is found the 
scanner agent starts up a corresponding agent which will be responsible for 
that annotation in the HTML file. If there is already an agent responsible 
for that piece of information, the scanner agent will skip the annotation. 
Changes to existing annotations (type of the agent or parameters) will cause 
the previously started agent to be killed and a new agent to be started 
up instead. The scanner agent parses a hierarchy of HTML files, starting 
up the team of agents that were specified by the Webmaster to look after 
the Web site. The Webmaster will include an annotation everywhere in the 
Web pages where there is a need for monitoring and updating of information 
and/or formatting. A special agent, the publisher agent is started up for each 
page with at least one annotation. This agent is responsible for collecting the 
updates on the pieces of information within its page and update them in the 
HTML file. The scanner agent does not abort when it finds ill-constructed 
annotations, but skips over them and tells the Webmaster about them. The 
scanning process does not check for the correctness of the HTML contents, 
simply concentrating on the annotations and their associated effects (i.e., 
start up of agents). 

Publisher Agents - The scanner agent starts up a publisher agent for each 
HTML file with at least one annotation. This publisher agent is responsi- 
ble for collecting the information from other agents looking after particular 
portions of the file and updating it. An alternative to our approach would 
be to have one agent looking after all annotated pieces of information of a 
page as well as updating the HTML file. However, if the annotated pieces of 
information have different frequencies for updates, then this one single agent 
would need to assess the time elapsed for each previous update in order to 
perform the next update. Although this is not an impossible task, such an 
agent will be unnecessarily more complex. 

Content Agents - For each piece of information annotated, a correspond- 
ing content agent is created. This agent’s task is to access periodically the 
data source as defined by the task frequency checking for any update in the 
information content. Having a specific agent for each piece of information 
isolates the details of the access to each data source, supporting multiple 
data sources and facilitating changes in the data source of any information. 
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Hence, content agents manage and access data only and publisher agents 

manage the Web pages visualisation (HTML files). 

3.2 A Team of (Logic-Based) Agents 

Each annotation may give rise to one or more agents, if the piece of information 
has not already got an agent started up. All our agents are self-contained pro- 
cesses with which Webmasters can communicate via message passing. The type 
of agent specified in the annotation informs the scanner agent which contents 
(functionalities) the agent ought to have. Each type is associated with a file con- 
taining the source code the agent will use once started up. The parameters in 
the annotation are passed to the agent which will use its source code with them. 

In principle, any programming language can be used to represent the code. 
We have employed a simple executable temporal logic called TeLA (Temporal 
Logic of Assertions) [8] that confers a clean design on our agents and makes 
it possible to formally verify them with temporal theorem provers [9] as well 
as with model-checking tools such as SPIN [10] and LTSA [11]. The temporal 
dimension is also required to cope with the issues of frequency. We have used 
SICStus Prolog [12] to implement all our agents and infrastructure. 

Our simple executable temporal logic has only one operator, the next time 
operator 0> and our formulae have a very restricted syntax, of the form Present 
Conditions => O Assertions, the meaning of which is, if it can prove PresentCon- 
ditions, a conjunction of non-temporal (ordinary) literals, at the current state of 
affairs (current state of the system), then Assertions , a conjunction/disj unction 
of (possibly temporal) literals will be made to hold in the next state. 
This simple tempo- 
ral logic is a prac- 
tical compromise be- 
tween the expressive- 
ness of full first-order 
temporal logic [13] and 
an efficient means to 
compute models for its formulae. The creation of such models guides the ex- 
ecution of temporal logic programs. We show in Fig. 3 a simple agent in TeLA. 
The execution changes between two states, wait and process(Msg), where Msg 
is a variable (we adopt Prolog [14] conventions) to be instantiated to a message. 
Our simple agent switches between these two states: it stays in the wait state 
until a message Msg arrives (predicate check succeeds if a message is available for 
the agent). When a message arrives it moves to state process(Msg) (first axiom) 
- a state in which the agent will handle the message it just got. The second 
axiom allows the agent to reply to the message it has received and the agent 
goes back to the wait state. 

We have enclosed the left and hand sides of the formulae in boxes to improve 
visualisation. The formulae show an agent that keeps an account of its current 
state, one of the set {wait, process {Msg)}, and the conditions that ought to 
hold in order for the states to change. Actions can be conveniently encoded as 





state(wait) A check(Msg) 


=> o 


state ( process (Msg)) 












state(process(Msg )) A reply(Msg) 


=> ° 


state(wait) 





Fig. 3. Sample TeLA Agent 
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conditions: when a predicate like check /I is attempted to be proved, it causes 
the execution of a routine to check if a message is available for the agent. A 
set of temporal formulae such as those above is put to use by means of a proof 
procedure that tries to build a model that satisfies all the formulae. The formulae 
depict the program and the proof procedure its interpreter. When agents are 
started up their corresponding program is the union of temporal formulae and 
the proof procedure. 

The proof procedure works by checking if the atomic formulae on the left- 
hand side hold true. If they do, then the right-hand side is used to build a model 
for the next state of the computation. The proof procedure builds a model for 
the next state in which all formulae hold. In order to prove the left-hand side, 
the proof procedure checks the model for the current state for all those atomic 
formula that hold. However, the proof procedure also keeps a conventional (atem- 
poral) logic program in which auxiliary predicates are defined - in our example 
above, the check and reply predicates are proved by means of one such pro- 
gram. The separation between temporal and atemporal aspects provides neater 
programs. 



3.3 Communication Among/with Agents 

Our agents are started up and run as independent Prolog processes which ex- 
change messages by means of the Linda process communication mechanism avail- 
able in SICStus Prolog [12]. Linda [15] is a concept for process communication, 
consisting of an area, the tuple space (shown in Fig. 1) where data from a num- 
ber of different processes can be stored and a few basic operations (write, read 
and delete) can be performed on this area. The Linda concept has proven very 
successful due to its simplicity and flexibility, being incorporated into a number 
of programming languages, including Java [16]. SICStus Prolog incorporates a 
Linda library and offers a repertoire of built-in predicates with which one can 
write programs that run independently and exchange data. The messages our 
agents exchange via the tuple space are variants of the FIPA-ACL [17] standard 
adapted to Prolog’s syntax. The information within messages are XML con- 
structs [7] transferred as strings. This standardisation allows for changes in our 
infrastructure: for the sake of simplicity, we have used Prolog to implement all 
components of our architecture. However, we could move to a more standard and 
popular multi-agent platform like JADE [18] without much reimplementation. 

Webmasters can communicate with agents, to find out about their status 
and follow their activities. A simple interface allows communication between 
the Webmaster and the team of agents. When agents encounter problems they 
can send messages to the Webmaster alerting to the difficulties they meet and 
whether they require intervention. Ideally, such interactions should be kept to 
a minimum, conferring as much autonomy to the agents as possible when they 
are being designed. 
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4 Working Example 

In this Section we describe three main- 
tenance agents as an example of the 
use of our approach for automated Web 
site maintenance. In this example there 
is one publisher agent and two con- 
tent agents. For simplicity, it is assumed 
that all Web site content is kept in a 
database and all agents have access to 
this database. Let us consider a Web site 
for weather forecast having the follow- 
ing information for each location: day of 
the week, minimum temperature, max- 
imum temperature and current temperature. This database can be repre- 
sented by the Prolog [14] facts weather (Location, Day ,MinTemp ,MaxTemp) 
and current jtevrpiLocation, CurrentTemp) . The information content in this 
database is constantly updated from multiple sources. The agents’ task is to 
keep this information updated on a page for a chosen location. Figure 4 illus- 
trates a Web page of this sort. Given that this sort of information is volatile, 
the content agents need to check the database periodically - for example, every 
hour the database is checked for a new current temperature. The forecast for 
maximum and minimum temperatures for the next 5 days does not change in 
the same frequency, for that reason there is another content agent responsible 
for keeping track of that particular information. 
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Fig. 4. Weather Forecast Page 



<html><head><title>Weather Forecast. . .</titleX/head> 
<body> 

<! — <agent info="days_temp" type="weather_agl" 
param=" [every, 12, hour] "> — > 
<table><tr><td>Monday</tdXtd>Tuesday</td> . . . 
<tr><td>33</tdxtd>31</td> . . . </table> <! — </agent> — > 
<br> Current Temperature 

<! — <agent info="curr_temp" type="weather_ag2" 
param=" [every, 1 ,hour] "> — > 

30 <! — </agent> — > °C ... 



Fig. 5. Annotation for Web Page Maintenance 

In order to specify the content agents, the portion in the Web page code where 
the information about current temperature and the 5-day forecast appears must 
be annotated as presented in Fig. 5. These are identified, respectively, as agents 
weather_agl and weather. ag2. Note that a publisher agent is automatically 
created for every page with an annotation. In this example we will identify this 
agent as pub.ag. 

A typical content update agent behaviour can be illustrated by a state tran- 
sition diagram, depicted in Fig. 6. From the initial state (start), the agent 
immediately moves to the check state, where it checks the data source for an 
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information update. At this point there are two possibilities: there is an update 
to perform and the agent moves to state update, or there is no update to be 
performed and the agent goes to state 
sleep. Yet another possibility is a prob- ' 

lem state, caused by unavailability of the 
data source, for example. In this case the 
agent should notify the Webmaster and 
finish its execution accordingly. We omit- 
ted this state for simplicity. 



► [update!— 



- sleep I 



Fig. 6. Content Agent Behaviour 



When an agent is in state update it puts the new information on the tuple 
space and notify all publisher agents that are expecting that message - we explain 
below how this notification is done. After sending the information, the content 
agent goes to sleep. The details of this coordination among agents is explained 
in Section 4.1. 



start O state (check) 

(state (check) A check_info(nil)) =>■ O state (sleep) 
(state (check) A check_info(I) A I 7^ nil) =>- 

O st ate (update, I) 

(state (update, I) A notify_publishers(I)) =>- 

O state (sleep) 

(state (sleep) A sleep(l ,hour) ) O state (check) 



State sleep cor- 
responds to a time 
(defined by the fre- 
quency of the task) 
in which the agent re- 
mains inactive. When 
the agent gets active 
again it moves back 

to state check. Fig. 7 Fig ' 7 ‘ Content Agent for Current Temperature 

shows the specification of a content maintenance agent using TeLA. Predicate 
check_info(I) encapsulates all necessary steps to access the data source and 
retrieve the latest data. If there is no new information available, the predicate 
returns nil. Predicate notify_publishers (I) makes the updated information 
available to the publisher agent. Predicate sleep makes the agent inactive for a 
specific duration of time, which is also specified in the Web page annotation. 



The specification of the five-day forecast agent weather_agl is basically the 
same. The only differences are the piece of information the agent is interested in 
and the frequency of its update. In this case the agent looks after the information 
with label days_temp, defined in the page annotation. 



The publisher 
agent is, however, 
rather different as 
it needs to know 
about the Web 
page visualisation 
style, the actual 
page layout and & & 

the specific place where the information managed by the content agents will 
be published. It works by placing an information request on the tuple space 
and it waits for a signal from the content agent responsible for that piece 
of information. Once the content agent has placed an updated information, 



start O state (check) 

(state (check) A in(inf orm(_,pub_ag,wake_up) ) ) =>- 

O state (publish) 

(state (publish) A get _all (Request) A 
extract_info (Request , Info) A publish(Inf o) ) => 

O state (check) 
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the publisher receives a “wake up” signal and updates the Web page. Fig. 8 
shows the publisher agent specification. In state check the publisher agent 
waits for a “wake up” message from a content agent, denoted by predicate 
in(inform(_,ag_pub,wake_up)). In state publish the publisher gets all 
updated information it needs via predicate get_all (Request) , then it extracts 
the value of the pieces of information from the messages using predicate 
extract _info/2, and finally publishes the Web page via publish(Inf o) . 
Details of the implementation of these predicates and those predicates used in 
the content agents are given in Section 4.1 below. Note that the page layout is 
associated with the publisher agent. It is encapsulated by predicate publish(I) 
which inserts the updated piece of information I in its right place as defined in 
the annotation. 

Changing the style is also possible, via the publisher agent. This is pos- 
sible because of an important feature of our approach: keeping content and 
visualisation management separate. We can define individual styles for pieces 
of information as a set of predicates where, given a piece of information, 
it produces a corresponding publishable piece of code in a target mark- 
up language, such as HTML. For example, itemize (L) where L is a list 
of items with the form [itemi , itemo , ..., item,,] results in the HTML 
code <ul><li>itemi</li><li>item 2 </li>. . . <li>item„ </li></ul>. Sim- 
ilarly, other style predicates are defined as table (LL), paragraph (P) , 
enumerate (L) , bigText(P), and so on, each resulting in a piece of code which 
is placed in a page template. As a result the visualisation style of a particular 
data is changed. This allows the publisher agent to change a piece of information 
presentation style without affecting its content and completely independent of 
content maintenance agents and data sources. This also reinforces the idea of 
separation between information content and visualisation details. 

Another use of the publisher agent is to re-publislr a corrupted or deleted 
page, as the agent includes the layout and all static information of a Web page. 
The content maintenance agents provide the dynamic information of a page and 
once with all this information a new page can be produced. 

The agents presented in this example keep the Web page presented in Fig. 4 
automatically updated. Whenever the current temperature or the 5-day weather 
forecast changes in the database the content agents capture the new values and 
send them to the publisher agent, which in turn updates the HTML file. 



4.1 Agent Coordination 

The maintenance agents are coordinated in a particular way, in order to provide 
independence between content and publisher agents, allowing the same piece 
of information to be used by more than one publisher agent and to minimise 
overhead in message exchanging. 

A major concept used in our agent architecture is the tuple space. It works as 
a notice board where any agent can post and retrieve messages. Our coordination 
mechanism is defined by this order of events: 
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1. the publisher agents put a request for updated information on the tuple 
space; 

2. the content agents, when they have new updates, look up the tuple space 
for requests for their pieces of information - each piece of information is 
identified by the label provided by the Webmaster; 

3. the content agents add their new information to the requests and put a wake- 
up notice on the tuple space to alert the publisher agents there is something 
for them. 

The request message format is request (Sender , Receiver , Info) :Flag. In this 
notation, Sender and Receiver are agent identifications. Info has the form 
inf o (Label .Data) , where Label is a unique identifier of a piece of information, 
as defined in the page annotation and Data is its corresponding value. The 
information label (originated in the Web page annotation) identifies a piece of 
information and is used by publisher agents to find the right place of the data on 
its Web page. Flag is used as an indication whether the information value Data 
has been updated by a content agent. Our convention is to set Flag to value 0 
indicating that Data has not been updated yet and zero otherwise. 

The second type of message is only 
a signal used to wake up publisher 
agents with the form inf orm(Sender , 

Receiver, Signal). This message is 
necessary to avoid unnecessary re- 
peated verifications of the tuple space 
by publisher agents, checking for up- 
dated information. Fig. 9 illustrates 
the given example showing how agents 
communicate via the tuple space. The 
top half of the diagram illustrates the 
pub_ag writing on the tuple space the 
requests for the two pieces of informa- 
tion it needs in its Web page - the en- 
tries have (anonymous variables) in the fields whose value is to be supplied 
by a content agent. 

The bottom half of the diagram of Fig. 9 depicts the content agents 
weather_agl and weather_ag2 independently accessing the tuple space and al- 
tering the requests with their information. The first agent to supply information 
to the publisher agent pub_ag also inserts the wake-up notice. In our example, 
weather_ag2 does this. 

The agent implementation in TeLA presented in Figs. 7 and 8 makes use of 
auxiliary (atemporal) predicates which actually perform the maintenance tasks, 
such as accessing a database, checking messages on the tuple space and creating 
the HTML files. It is important to point out that these predicates also have the 
important role of hiding implementation details, keeping the agent specification 
clean and easier to read. It allows, for instance, changing the access to data 
sources, without changing the main specification in TeLA. We describe the im- 
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plementation details of these predicates below. The content agents also include 
three such predicates: 

1. check.inf o/l: succeeds if the data item has changed in the data source, re- 
turning its value, or succeeds if the data item has not changed, returning nil. 
The actual implementation might involve details for connecting and accessing a 
database, via SQL queries, for example. 

2. sleep/1: succeeds if the execution of an agent is suspended for the duration 
specified by the arguments (number and time unit). 

3. notify_publishers/l: succeeds if there are requests from publisher 
agents for the piece of information in the tuple space and sends the 
updated value of the information and a signal to wake up the pub- 
lisher agents who have requested that information. The code in Fig. 10 
illustrates this predicate implemen- 
tation for the agent that updates 
the current temperature. Predicate 
bagof_rd_noblock(A,B,C) builds 
list C with all terms A that match 
term B in the tuple space. In the 
program in Fig. 10, AllRequest is a 
list of publisher agent ids, who have 
asked for information curr_temp. 

Publisher agents also require special 
predicates described in [19]. 

5 Related Work 

We can speculate that batch files and daemons have been employed in an ad-hoc 
fashion by Webmasters to automate repetitive tasks, but these have not led to a 
systematic approach to Web site maintenance. One major drawback with batch 
files and daemons is their lack of responsiveness: in order to find out about them 
(i.e., whether they are alive and running), the Webmaster must look up their 
process identifications and their status or log files - scaling up clearly becomes 
a problem. 

Similar annotations, the associated scanning process and the bootstrapping 
of agents were originally described in [20], using Java [21] as a programming 
language and JADE [18] as a communication platform. Each agent incorporated 
a particular functionality: a particular piece of information from another Web 
site was specified for the agent to monitor and periodically fetch, using it to 
update one’s Web site [22]. These two references inspired our work herewith 
described and, apart from them, we have not been able to find any work on 
using multi-agent systems to look after Web sites. 

A number of methods and techniques for Web site development have been 
proposed such as OOHDM [23], Araneus [24], Strudel [25], WebML [26], UWE 
[27], and the logic-based approaches of [28] and OMSwe [29], among others. The 
use of a formal approach for Web site modelling and development facilitates 



notify_publishers(I) 
bagof _rd_noblock (P , 

request _inf o (P , _ , inf o (curr.temp , : 0 , 

AllRequest) , 
my_id(Me) , 

send.info (Me, AllRequest ,1) . 

send_info(_, [] . 

send.inf o (Me , [P I RestP] ,1) : - 

out (request _info(P, Me , inf o(curr_temp, I) ) : 1) , 
out (inform (Me, P, wake _up) , 
s end JLnf o (Me , RestP , I ) . 



Fig. 10. Content Agent Implementation 
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maintenance tasks as they usually provide a high-level view of the application 
and separation between content, navigation structure and visualisation. This 
allows, for instance, updating page templates or colour schemes without affecting 
the page content or links. Although these works have addressed the problem of 
Web site application development in the large, where maintenance is one of the 
issues involved, our work proposes an approach to static Web page maintenance 
which can also be adapted in order to be employed as part of a complete Web 
site application development method, as those mentioned above. 

Specific automated maintenance is not addressed by most methods, although 
some claim support for it or offer some degree of automation. Given that there 
are certain maintenance tasks which are well defined and repetitive, it is possible 
to automate them in order to avoid human intervention, saving maintenance time 
and keeping the Web site continuously updated. With this respect, two work are 
closer to our proposed approach. 

Sindoni [30] proposes au approach to enforce consistency between page con- 
tent and database state. A specific algebra is proposed for defining views and 
views updates, which uses the Araneus Data Model as the reference model. 
Views updates are used by an algorithm that automatically updates the Web 
site from a set of changes on the application database. Maintenance is performed 
by removing or generating the appropriate sets of pages. 

OntoWebber [31] is an ontology-based approach designed for the generation 
of data-intensive Web sites, in particular Web portals. It addresses the prob- 
lem of multiple and heterogeneous sources of data by defining integration and 
articulation layers responsible for resolving syntactic and semantic differences 
between the different data sources. The composition layer includes a specifica- 
tion of a Web site view based on the Web site modelling ontologies. Ontologies 
are described using DAML+OIL [32] , a semantic mark-up language for Web re- 
sources. A generation layer process queries the site specification to produce the 
Web pages. This approach offers some degree of automation by means of rules 
defined as triggers. These rules update the source data, meta-data, and site view 
specifications according to the fired triggers. However it is restricted to content 
maintenance. 



6 Conclusions and Discussion 

In this paper we introduce an agent-based approach to the maintenance of Web 
sites. Our approach caters for static Web pages whose contents require moni- 
toring and updating. We offer a notation that Webmasters can use to annotate 
particular points of a Web page (an HTML file) in order to specify that an agent 
should monitor and update that portion of the page. The Webmaster can add as 
many annotations as required in one Web page. Our architecture associates, via 
a scanning process (carried out by an agent itself) an agent to each annotation. 
This agent is started up and autonomously looks after its prescribed piece of 
information. Any modifications to a Web page are centrally carried out by a 
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publisher agent with whom all agents communicate - each annotated Web page 
has a publisher agent, started up by the scanning process. 

It is important to decouple our proposed architecture and its implementation, 
as explained here. We do not suggest that ours is the only or the best means to 
implement the proposed architecture: our implementation should be considered 
as a proof-of-concept prototype used to exploit and refine our ideas. However, 
our implementation captures all basic functionalities of our architecture and is 
evidence that our architecture is feasible. Some features of our proposal worth 
pointing out are: 

— Scalability - as many agents as required can be started up, provided that 
there are available computational resources. In our experiments, we used up 
to 250 agents in one single Pentium III PC (1GHz) running under Linux. 
However, the scanner agent can be programmed to start up agents in differ- 
ent machines. The SICStus Prolog tuple space employs a unique hoskport 
addressing which allows agent anywhere in the Internet to access it. 

— Ease of use - agents are now responsible for tasks that relied on humans or 
off-line daemons and batch files. Rather than keeping a record on the status 
of daemons and batch files or the actions of humans, the manager can now 
communicate with thousands of agents collectively or individually. 

— Robustness - because the task of monitoring and updating the pages of the 
Web site is divided among independent agents, if individual components fail 
it is still possible to achieve some degree of overall functionality. Additionally, 
we can enable agents to perform failsafe operations when they encounter 
exceptional circumstances. For instance, if the data source an agent is using 
suddenly becomes unavailable, the agent could provide a “Not Available” 
default value to update the information on the Web page. The agent could 
also send a message to the Webmaster and wait for further instructions. 

— Backwards compatibility - any existing Web site can incorporate our archi- 
tecture, as the annotations are naturally accommodated within HTML files. 

— Extensibility - the class of available agents and their functionalities can be 
gradually extended, and new annotations specifying them can be added at 
will. Web pages can be gradually annotated as the Webmaster becomes used 
to the new managerial style of administering a team of software agents. 

Hyperlinks within Web sites may need constant monitoring as the objects they 
point at may move or disappear - we envisage employing software agents for 
constantly scanning the whole collection of HTML files, checking for broken ref- 
erences. These agents notify the Webmaster and isolate the offending reference, 
wrapping it as a comment. We are currently working on how these agents can 
be incorporated into our architecture. 

Work is under way to integrate our proposal for agent-based maintenance 
with a high-level specification of Web sites, as described in [28,33]. This ap- 
proach also uses logic to specify a Web application thus facilitating the desired 
integration. We notice that in a high-level specification of a Web site application 
the annotations for associating pieces of information to agents do not need to 
be made in the HTML files directly; rather they should become part of the site 
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specification. This opens new possibilities to improve both approaches to Web 
site synthesis and maintenance. 
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Abstract. Development of Web applications is dynamic by its very nature. Web 
development processes have to facilitate a Web application's continual 
refinement and evolution based on feedback front end-users. Evolutionary 
development can easily be achieved by end-user involvement through seamless 
integration of feedback and issue reporting mechanisms into Web applications. 
This paper discusses the use of conventional methods and tools for maintenance 
and change management as an infrastructure for evolutionary development of 
Web applications. An example demonstrates the feasibility of the proposed 
approach. It describes our experience from integrating the open source issue 
tracking system Bugzilla into a Web application. 



1 Introduction 

Today's economic realities pressure organizations to continuously adapt to shifting 
environments. Accordingly, "systems should be under constant development, can 
never be fully specified and are subject to constant adjustment and adaptation" [1], 
"Web developers have the capability to modify their systems for all users imme- 
diately, without being impeded by the manufacturing, distribution and sales channel 
delays inherent in shrink-warp software development." [2] Web applications are 
installed on a central server and modifications are instantly propagated to all users. As 
a consequence, Web applications can be developed in an evolutionary process. Once a 
feature is implemented, end-users can start using it and their feedback can be quickly 
incorporated into new releases. This iterative process of delivering the application in 
multiple small steps allows immediate response to rapidly changing business needs 
while continually maturing and adapting the application over time. Refinement and 
adaptation are in many cases the only viable option in view of the fact that business 
needs often change as development proceeds, making a straight path to an end 
product unrealistic [3], 

The goal of this paper is to present an easy yet effective approach to seamless end- 
user involvement into the evolutionary development process of Web applications. By 
relying on existing maintenance infrastructure, the users' feedback can be routed 
directly to the developers with little additional overhead. According to this goal, the 
remainder of this paper is structured as follows. In section 2 we present requirements 
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for a Web development process. Section 3 outlines our conventional maintenance ap- 
proach and the associated infrastructure (methods and tools), while section 4 
describes how we enhanced the maintenance infrastructure to support evolutionary 
Web development. Section 5 gives an overview of related approaches that deal with 
rapid changes in Web development, and section 6 concludes the paper. 



2 Requirements for a Web Development Process 

From our experience in developing Web applications as well as from literature 
research, we have derived a list of requirements for our Web development process. 
The most important requirements are to provide (1) end-user involvement, (2) 
prototyping, (3) change management, (4) immediate response, (5) risk minimization, 
(6) no administrative overhead, and (7) transparency and overall guidance. 

End-user involvement. Among the top three reasons for challenged or failed 
projects is the lack of end-user involvement [4], Knowing the end-users' requirements 
is essential for the development of successful Web applications. The customer, 
although defining the main goals for the development of a Web application, usually is 
not the actual end-user and, therefore, he or she is not able to define all the require- 
ments important to end-users. Involving actual end-users is most effective when un- 
certainty about requirements is high [5], but difficult to achieve in Web projects. The 
main reasons are the potentially high number of end-users, their geographical 
distribution (possibly all over the world), and their anonymity due to the lack of direct 
interaction. 

Prototyping. End-users should be capable to provide all necessary requirements. 
However, as Web applications and their related concepts and metaphors are still new 
to many users, they have difficulties to develop realistic expectations and to express 
their needs [6]. Prototyping is used to leverage the involvement of end-users in Web 
application development. End-users are potentially highly skilled evaluators of 
product functionality [7], Thus, we find it viable to prepare a first, stable prototype 
and present it to the end-users as a basis for feedback. Thereby, the users can refer to 
a clearly defined part of the application, which helps to avoid a plethora of 
(unrealistic) ideas and wishes. With emphasis on evolutionary prototypes of 
reasonable usability and reliability, we strive to establish continuity in development 
and avoid confusion and frustration of end-users - the main pitfall of rapid application 
development in Web development according to [8]. 

Change management. In many cases, it is not possible to fully specify the 
requirements of a Web application at the beginning of the project, because they will 
either evolve over time or change during development. Typical reasons are frequent 
changes in the environment (e.g. new business partners and competitors), new user 
requirements (stemming from intermediate development results presented to the user), 
the availability of new technologies, methods, and tools, or the need for refactoring 
and error correction throughout development. Thus, the incorporation of effective 
mechanisms to manage the Web application's change and maintenance is one of the 
key steps of successful development [9]. A single channel for requests and issues has 
to be defined and a change control board to prioritize requests and issues has to be 
established to allow for flexible and prompt reaction. 
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Immediate response. Developing Web applications means “internet-speed” 
software development [10]. Immediate response to changing requirements, to user 
feedback, and to market shifts is required. Regarding the aforementioned change 
management, immediate response relies mainly on immediate reaction to feedback 
from the end-users to intermediate development results, such as prototypes. Thereby, 
immediate feedback to the user is necessary in order to inform the end-user about the 
consecutive steps and to show that his/her effort is taken seriously, even if not all 
suggestions can be incorporated right away. In addition, immediate response to 
market situations provides the ability to tackle the demanding time-to-market problem 
in Web development. By employing evolutionary prototyping [7] it is feasible to go 
on-line with a first working solution that serves as a starting point for further 
development efforts and to exploit the opportunity of being a first-mover. 

Risk minimization. Web projects have to deal with a high level of uncertainty and 
a large number of risks [12]. Typical examples are unrealistic schedules, misunder- 
stood or frequently changing requirements, underestimated complexity of non-func- 
tional requirements, and unstable technologies. An ideal Web development process 
explicitly addresses these risks. Therefore, a realistic estimation of the project situa- 
tion is necessary. This is possible through frequent feedback from customers and end- 
users on working examples and the continuous adaptation of previous estimations. 

No administrative overhead. Typical Web projects have to cope with harsh limi- 
tations of financial resources, and - in addition - with restrictive schedules, often less 
than one or two months [11]. Thus, it is important to keep the administrative overhead 
at a minimum. Lightweight or agile approaches have to be applied and the often im- 
mense burden of (inappropriate) regulation and tool usage has to be avoided in order 
to keep both process costs and reaction time low. Yet, lightweight tools can signifi- 
cantly increase effectiveness and efficiency with simple, problem-specific solutions. 

Transparency and overall guidance. In evolutionary Web development estab- 
lishing a clear vision and long-term goals for the project is essential to unite the 
project team's efforts and to assure that all the efforts are aimed in the right direction. 
The product management, comprising the main stakeholders (e.g., customer and 
project manager), is responsible to outline a roadmap for future work and to align the 
project goals with business goals. A shared vision is required. Misalignment with 
business goals is quoted as one of the ten deadly Internet and Intranet project risks 
[12] . The collected and reviewed feedback from end-users serves as a decent 
controlling instrument. For effective guidance, the whole process has to be transparent 
for the team members as well as for the customers. 

Most of these requirements have been frequently cited throughout the Web 
engineering literature ([8], [13], [14], [15], [3]) as prerequisites for the development 
of Web applications. For us, the question is not whether or not to incorporate these 
requirements, but how we could do this without the burden of changing existing 
processes and established structures or applying new methods and new tools. Yet, a 
rigorous reduction of development cycles resulting in a blend of development and 
maintenance activities proved as an effective and pragmatic approach towards an 
evolutionary development process. Tool support helped to make this approach 
efficient and systematic. 
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3 (Conventional) Maintenance and Change Management 

Maintenance is the modification of a software product after it has been placed into 
production. Commonly, four different types of maintenance are distinguished [8]: (1) 
Corrective maintenance to fix bugs and design deviations, (2) preventive maintenance 
to avoid future bugs or maintenance, (3) adaptive maintenance when a system's en- 
vironment changes, e.g. when a new Web browser is released and the application has 
to be modified to work properly within that browser, and (4) perfective maintenance 
to introduce enhancements such as new functionality or to increase efficiency. 

Keeping track of changes and their effects on other system components is not an 
easy task. The more complex the system is, the more components are affected by each 
change. For this reason, change management [3] - important during development - is 
critical during maintenance. For conventional software development, our approach is 
to establish a change management process lead by a change control board to oversee 
this process. Open source tools, such as CVS and Bugzilla, are employed to support 
control and collaboration among team members. 

CVS (Concurrent Versions System) [16] is used for version control. This tool 
keeps track of the different versions of all the files that are part of a project. CVS 
version control significantly eases the collaboration between developers and the 
authoring of Web sites [17). In addition, we rely on Bugzilla [18] to report and track 
issues (e.g. requirements, bugs, user requests, or remarks). Bugzilla is a popular open 
source issue tracking system that derived from the open source project Mozilla. It 
provides an extensive set of features useful for distributed development teams and 
end-user involvement, such as a Web-based user interface or an email notification 
mechanism. As the source code is freely available, Bugzilla can easily be adopted and 
extended to meet specific organizational needs. Nevertheless, the predefined 
workflow introduces a flexible yet clear and systematic way to deal with issues. The 
state diagram in Figure 1 illustrates the basic steps of the workflow from the per- 
spective of an issue's lifecycle. Further details can be found in [18] or [19]. 




Fig. 1 . Bugzilla issue management workflow 
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4 Transition to Evolutionary Development 

David Lowe compares the development of Web applications with town planning and 
landscape gardening: "The evolution of Web applications is analogous to a garden 
changing as a natural part of its cycle of growth." From this, Lowe draws the 
following conclusions: "... Web development tends to differ greatly in that we are no 
longer aiming to develop a 'finished' product. Rather, we are aiming to create an 
organic entity that starts with an initial consistent structure, but continues to grow and 
evolve over time. This evolution is much finer-grained than the maintenance changes 
that occur with more traditional software products, and tends to be an integral part of 
the lifecycle of the product. Compare this to conventional software maintenance, 
which tends to be a coarse-grained response to errors in the products, changes in 
requirements, or a changing environment." [20] 

In conformance to Lowe's suggestions, we reviewed and aligned our development 
process. Instead of aiming towards a final product from the very beginning, we started 
with laying out a general, flexible fundament for a more generic Web application by 
modularizing functionality and by relying on component-oriented development. For 
our first projects, we used an in-house Web application framework, similar to the 
architecture described in [21], since other available solutions were still in their 
infancies. The framework allowed us to develop modular function blocks and to 
specify their coupling, parameters, error conditions, and access rights through XML 
configuration files. This framework was later on superseded by the freely available 
Struts framework [22], part of the Apache project. On the basis of this initial structure 
we developed the functionality in small increments to enable evolutionary growth. 
These increments did not encompass fully featured implementations of functions, but 
rather initial, stable, and useable prototypes to demonstrate the conceptual and 
technical feasibility. Only the feedback from the end-users facilitated the function's 
continual refinement and maturation. 

In the remainder of this section we describe how we implemented such a feedback 
mechanism by means of the issue tracking system Bugzilla for a Web application for 
internal use, the lessons we learned, and further implications. The experience and data 
presented in this section is a summary of the project's final report, the usage analysis 
of Bugzilla, and the notes from periodical project and process reviews conducted by 
quality management. 



4.1 Enabling End-User Involvement 

At a first glance, the approach to evolve an application based on user feedback does 
not seem much different to any conventional (iterative) software development process 
that includes a beta testing and stabilization phase. For us, this was an advantage as no 
major changes to the underlying processes and no new methods or tools were 
necessary. However, there exists an inherent difference regarding the nature of the 
feedback that is often overlooked, especially by development: A comparison of 
conventional and evolutionary development shows a significant shift in the 
importance of the different types of maintenance. While conventional maintenance 
approaches tend to emphasize corrective maintenance, evolutionary development is 
characterized by a focus on adaptive maintenance [23]. This is an important finding, 
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as it underlines that evolutionary development focuses on (new) requirements rather 
than on errors in existing functionality. Evolutionary development should therefore 
primarily address the problem that users are not able to fully specify their 
requirements at the beginning of the projects and, even if they do, requirements may 
still change over time as business procedures and technologies evolve. Furthermore, 
evolutionary development is often the preferable way to develop a common 
understanding about the goals of the project and the needs of the users [6], "In other 
words, the specification of the system emerges from the design, rather than preceding 
and driving the design." [24] 

Consequently, we used the maintenance infrastructure, namely the issue tracking 
system Bugzilla described in section 3, primarily to collect and maintain the 
continually evolving requirements of the application. Thereby, using the existing 
maintenance infrastructure for evolutionary development provided several benefits: 
First of all, a proven tool with a clear and systematic process was used. All the 
involved developers were familiar with the infrastructure. Thus, similar to the 
advantages initially mentioned, no radical changes of the underlying development 
process were necessary and overhead costs could be kept at a minimum. Second, as 
described below, the integration of the Bugzilla issue tracking system turned out as an 
effective vehicle to interact with end-users and to collect requirements. Third, it 
supported collaborative work throughout development as well as measuring and 
tracking the ongoing development effort. 

To establish a single channel for end-user feedback, we made our maintenance 
infrastructure an integral part of the Web application. Beside the solution presented 
here, we also experimented with various other approaches that are commonly 
suggested in the literature for end-user integration (see [23]). Useful to stimulate 
feedback and discussion are, for example, annotation mechanisms to Web pages or 
Wiki-style authoring [25]. However, these approaches focus mainly on content - the 
data, its structure and presentation. We required a more general feedback channel 
amenable to the different functional and non-functional aspects of Web applications 
(e.g., correctness, security, usability, or performance issues related to functionality, 
content, or infrastructure aspects) as described in [26]. Nevertheless, an easy to use 
and seamless integrated solution was required. When, in a first step, Bugzilla was 
integrated solely via a link, we experienced little acceptance of the feedback 
mechanism because of major usability deficiencies: The access to Bugzilla was rather 
uncomfortable due to switching to a (very) different application. The users had to 
logon again and were confronted with the (non-intuitive) user interface of Bugzilla, 
data had to be copied manually into Bugzilla's report form, and the users were 
burdened with the concepts of Bugzilla's issue tracking process. 

To overcome these drawbacks, we integrated a simple issue report and enhance- 
ment request form into the Web application, which forwarded the data entered by the 
user to the issue tracking system Bugzilla. Even though the form resembled the 
structure of a conventional Bugzilla issue report, it was aligned with the style of the 
Web application (see Figure 2 for an example). A few additional field values, e.g. 
enhancement request for the field issue type, had to be added to support not only issue 
tracking but also requests for enhancements and new features. Hence, the users were 
relieved from directly accessing the Bugzilla interface and we could still utilize our 
preferable issue tracking system in the background. Furthermore, the form was made 
available within a single click from every page of the Web application. On accessing 
the form, the context of use was analyzed and the fields of the form were 
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automatically pre-filled with meaningful default values to minimize the amount of 
data the user had to enter. For example, the user's browser type was identified to 
easily reproduce issues related to the different behavior of the various Web browsers 
and the link referring to the page plus the name of the application module the user last 
accessed were automatically suggested. 




Fig. 2. Issue report and enhancement request form 



To get a first version of the Bugzilla integration setup up and running, we relied on 
the Bugzilla email gateway. Technically spoken, the report form is submitted to a 
Java mailing servlet 1 part of our Web application. The servlet assembles an email 
containing the data from the report form structured according to the requirements of 
the Bugzilla email gateway and sends it to a dedicated email account on the Bugzilla 
server. There, the email is parsed, the report data is extracted and entered into the 
issue tracking system. Thus, we can easily access Bugzilla and initiate the issue 
tracking workflow. Bugzilla auto-assigns the report to the developer in charge for the 
affected module, tracks all changes, and sends email notifications on updates. 
Figure 3 illustrates the interaction between the user's Web browser, the Web 
application, and Bugzilla integrated via the email gateway. 



1 The Java source code of this servlet can be obtained from the authors. 
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Fig. 3. Bugzilla Integration in Web Applications 



4.2 Experience and Lessons Learned 

The experience and lessons learned we present here are derived from integrating 
Bugzilla as part of an internal Web-based application. The main results were largely 
equivalent to the experience we made in consecutive projects. The target audience of 
the application was a group of experienced users from within our organization as well 
as partner companies at distributed locations. 

The possibility to submit comments was well accepted by end-users. Even though 
no particular measures were taken to encourage user feedback, we received more than 
100 reports within one month from a group of about 50 potential users. Many of these 
reports would otherwise have been submitted via email or through informal personal 
communication, e.g. by phone or in hallway conversations. Or, more likely, they 
would have simply been omitted. The development team greatly benefited from the 
issue tracking capabilities of Bugzilla that supported a quick triage as well as the 
systematic management of the submitted reports. 

The total number of reports was distributed as follows: 17% critical errors, 41% 
errors of normal severity, and 42% improvement suggestions and enhancement 
requests. Critical and normal errors together made 58% of all collected reports, 
despite the fact that we estimated a much higher potential for improvement comments 
and enhancement requests than for error reports. The reason seems to be that the 
motivation to react and comment on problems increases with the perceived severity of 
the problem. From personal feedback we know that, in some cases, users even did not 
report obvious errors when they found easy workarounds. Thus, incentives may be 
necessary to motivate feedback and, furthermore, the costs for providing feedback 
must be kept at a minimum. The ease of use of the feedback mechanism is of 
uttermost importance. A short and concise form and a careful pre-selection of default 
values are required as lengthy forms to fill in deter many users. 

However, including user session data in reports may possibly conflict with privacy, 
a major concern of many users. Therefore, we used only a few pre-filled entries and 
we did not log any usage data related to the reported issues. Rather, the user was 
asked to give a description of the issue and the necessary steps to reproduce it. The 
optional statement of an email address allowed us to contact the user in case of further 
questions. Besides, we were able to offer the user email notification on certain events, 
e.g., when the reported issue had been accepted or resolved. So we involved the end- 
users into the development process instead of frustrating them with the feeling as if 
reporting "into a black hole". 
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Two important lessons we learned were, first, that we could not expect users to 
positively confirm correctly working solutions. To some extent, the absence of 
comments may only indicate a correct working solution. Second, since we provide the 
feedback form for existing functionality only, users did not come up with new 
features or any "great new ideas". Hence, evolutionary Web development still 
requires a great deal of planning ahead and the proactive development of new 
functionality, e.g. motivated by competitive analysis [2], In addition, careful analysis 
of the end-users' reports and a "creative mindset" are necessary to stimulate new ideas 
within the development team. A shared vision serving as overall guidance for the 
project team helped to direct these ideas into the right direction. 

Furthermore, reports have to be brought in relation to overall usage statistics to 
evaluate the relative significance of the feedback. Other sources that provide an 
insight into the attitude and behavior of users should be included in the data collection 
to support the reasoning about improvements and to strengthen the conclusions 
drawn. Therefore, we are currently extending our feedback mechanism to allow real- 
time analysis of application logs as described in the following section. 



4.3 Towards a Bugzilla Web Service 

The approach described in section 4. 1 allows a quick and easy integration of Bugzilla 
into any Web application. However, the Bugzilla email gateway is an extension to the 
Bugzilla project with some limitations regarding functionality and flexibility. It lacks 
full access to all of the features of Bugzilla. Currently, only a one-way access to the 
issue tracking system is possible - issues can only be submitted from the Web 
application to Bugzilla. Full access should permit creating new issues, adding 
comments, changing issue related metadata (e.g. priority or severity), querying for 
existing issues, or voting for issues (so the frequency of issues can be documented and 
the number of duplicate reports can be reduced). 

In Figure 4, we outline an integration concept based on a SOAP (Simple Object 
Access Protocol) interface [27] to Bugzilla. Thus, the SOAP interface provides access 
to all of the Bugzilla functionality via a Web Service and enables communication in 
both ways - from the Web application to the Bugzilla issue tracking system (e.g. to 
submit an issue) and from Bugzilla to the Web application (e.g. to return the status 
information of an issue). The example of an automated issue reporting system, part of 
a Web application, illustrates the advantages of such an interface to Bugzilla. We are 
currently using the Log4J framework [28] for application logging. So, a log 
recorder/analyzer can be used to observe and analyze the activity and state of the Web 
application and react to certain events, e.g. application errors, in real-time by 
submitting appropriate reports directly into the issue tracking system without further 
human interaction. 

We are currently evaluating the Jagzilla Web Service API, part of the Jagzilla 
System [29] as a way to realize an integration of Bugzilla into Web Applications like 
depicted in Figure 4. Jagzilla provides a simple re-implementation of the core 
functionality of Bugzilla while relying on the original database of the Bugzilla issue 
tracking system. By the date of writing, the Jagzilla Web Service API is still in early 
alpha status. 
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Fig. 4. Bugzilla integration based on Web Services 



5 Related Work 

Being in a constant state of flux, the adaptation and continuous evolvement of Web 
applications as an answer to rapidly changing requirements has long been a topic in 
Web engineering. Various approaches to deal with this problem have therefore been 
proposed. In this section, we give an overview of related work, which we consider as 
a prerequisite, a logical next step, or a useful basis in combination with the approach 
that has been described in this paper. 

Modeling and design. Web applications must be built with a mindset towards 
frequent changes throughout the lifecycle. Development has to establish a basis for 
applications that can be modified, fixed, or maintained at little cost of time or money. 
Common approaches are modular architectures, e.g. based on re-useable components, 
Web application frameworks, or code generation from domain models. Various 
design methodologies support these strategies. An overview of methods and 
appropriate tools specific for Web applications can be found in [30], [20], [23], or 
[31]. From the maintenance perspective, re-engineering and reverse engineering are 
of particular interest. Methods and tools based on modeling and design strategies have 
been developed that support maintenance and evolutionary development. (Prominent 
examples are STRUDEL [32], ReWeb [33], WARE [34], or Rigi [35].) Evolutionary 
prototyping-based development, furthermore, has a rich tradition in User-Centered 
[36] and Participatory Design [37], Participatory Design explicitly considers social, 
ethical, as well as political viewpoints in addition to technical issues and takes a 
"democratic" approach to system design by actively involving users. From our point 
of view, modeling and design techniques are a prerequisite for effective maintenance 
and, thus, complementary to our evolutionary development approach. 

Adaptive and customizable Web applications. With the transition to dynamic 
Web sites and Web-based applications, where Web pages can be generated on the fly, 
the idea of self-adaptation in response to external changes emerged. Several 
approaches have been published that demonstrate this idea. For an overview of this 
topic please refer to [23], [38], [39], or [40]. In contrast, our approach requires a 
human (e.g. a programmer or product manager) to look at all submitted feedback 
reports, to validate them, and to initiate proper measures. The necessary changes - the 
adaptation and the extension of the Web application - are done by development, not 
by the system itself. However, an integration based on Web Services, as described 
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above, may provide enough flexibility to serve as a first step towards adaptive 
applications. 

Agile development. Agile development methods have successfully found their 
way into Web development processes [41, 42] and, conversely, agile processes have 
been developed for Web engineering [43], There are a number of rapid development 
software methodologies all referred to as "agile development" [44]. Many of these 
methods explicitly address the requirements of an evolutionary Web development 
process as stated in Section 2. They offer, for example, rapid prototyping, 
comprehensive end-user involvement, support for frequent changes (e.g. by unit 
testing), and they emphasize low administrative overhead. As we do not prescribe any 
development process in our approach, the workflow and tools proposed in this paper 
may be combined with most of the agile development methods. Some processes, e.g. 
Feature-driven Development [45], even suggest similar ideas to cooperate with stake- 
holders or to organize work. 



6 Conclusions and Future Work 

Within this paper we first presented a requirement's perspective for an ideal Web 
development process. In order to address and fulfill the presented requirements, we 
then elaborated on how we successfully implemented a process for evolutionary de- 
velopment based on conventional maintenance concepts. Our pragmatic approach has 
the advantage that established development processes can be retained, no radical 
changes are necessary, and existing tools can be utilized. Furthermore, while mainte- 
nance such as adding new features or correcting errors may be initiated by the de- 
velopment team, in practice maintenance is often triggered by feedback, both direct 
and indirect, from end-users. The seamless integration of the issue tracking system 
Bugzilla into the Web application permits easy and flexible involvement of end-users 
and encourages the feedback necessary to adapt and grow Web applications. Thereby, 
the workflow implied by Bugzilla supports an ordered and systematic development 
process (as demanded in [9]), regardless of which particular process model is actually 
used. From our point of view, agile development processes conveniently harmonize 
with Web development and prevent ad-hoc approaches and undisciplined hacking 
(see also [46]). 

In a next step, we will consider to integrate the interface to the Bugzilla issue 
tracking system into an existing Web application framework (e.g. Struts). This would 
considerably ease the automated collection of end-user feedback and, thus, further 
support the development of evolving Web applications as described in this paper. 
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Abstract. The continuous advances in Web technologies are posing new 
challenges to Web Engineering proposals, which now require the inclusion 
Software Architecture techniques in order to integrate the explicit consideration 
of non-functional features in the Web application design process. In this article 
we propose a new approach called WebSA, based on the the MDA (Model 
Driven Architecture) paradigm. WebSA specifies a model driven process that 
adds to the traditional Web-related functional viewpoint a new software 
architectural viewpoint that permits, by means of successive model 
transformations, to establish the desired target application structure. 



1 Introduction 

The high pace at which advances in Web technologies are taking place has changed 
the idiosyncrasy of Web applications, that now imply not only more complex 
functional requirements but also stricter constraints posed on features such as 
distributability, scalability, platform-independence or extensibility. In order to tackle 
such new requirements, several authors [1] propose the use of Software Architecture 
techniques. Following this trend, in this article we present WebSA (Web Software 
Architecture). WebSA is a Web model-driven approach that is based on the standard 
MDA (Model Driven Architecture) [5]. The MDA framework provides WebSA not 
only with the possibility to specify and formalize a set of Web-specific models by 
means of a UML profile, but also to specify each process step from the models to 
implementation by means of a set of transformation rules. 

The remaining of the article is structured as follows: in section 2 we present briefly 
the WebSA Development process and its main views and models. From these views, 
the architectural viewpoint and, more specifically, its logical architectural view is 
discussed in section 3. Last, section 4 outlines the conclusions and further lines of 
research. 



2 WebSA: Model Driven Architecture of Web Applications 

As we stated above, WebSA is a proposal whose main target is to cover all the phases 
of the Web application development and to contribute to cover the gap existing 
between traditional Web design models and the application implementation. In order 
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to achieve this aim, it defines an instance of the MDA Development Process for the 
Web application domain. Also, it proposes the formalization of the models by means 
of a MOF-compliant repository (metamodel) and a set of OCL constraints (both part 
of the OMG proposed standards) that together specify (1) which are the semantics 
associated with each element in the models, (2) which are the valid configurations and 
(3) which constraints apply. This formalization is being performed, as other Web 
methodologies [14] have done before, by the definition of a UML [8] profile that 
gathers the main constructs of the models involved in the specification of applications 
in the Web domain. 

In order to define a Web application System proposes a Web application view model 
that is made up of 8 views, grouped into three viewpoints: requirements, functional 
and architectural viewpoints. From them, the architectural viewpoint is a main 
contribution of WebSA. This viewpoint includes a logical architectural view that 
gathers the set of logical components (subsystems, modules and/or software 
components) and the relationships among them. Also, it includes a physical 
architecture view that describes the physical components that integrate the lower level 
specification of the application (clients, servers, networks, etc.). As we have stated 
above, and in order to shift from one view to the other, WebSA defines a process that 
is explained next. 



2.1 The WebSA Development Process 

The WebSA Development Process [3] is based on the MDA development process. In 
order to fulfill this goal, it establishes a correspondence between its Web-related 
artifacts and the MDA artifacts. Also, and as a main contribution, it defines a 
transformation policy partly driven by the architectural model that can be seen in Fig. 
1. In this figure we observe how in the analysis phase the Web application 
specification is divided horizontally into two viewpoints. The functional-perspective 
models reflect the functional analysis, while the architectural models define the 
system architecture. Both of these models are PIMs in the context of an MDA 
framework. In this phase, the architectural models are based on the concept of 
Conceptual Architecture [4], and are made up of conceptual elements, obtained by 
abstraction of the elements found in the Web application domain. These models fix 
the application structure orthogonally to its functionality, therefore allowing their 
reuse in different Web applications. 

The PIM-to-PIM transformation (see T1 in Fig. 1) of these models into platform 
independent design models (PIMs) provides a set of artifacts in which the conceptual 
elements of the analysis phase are mapped to concrete elements where the 
information about functionality and architecture is integrated. It is important to note 
how these models, being still platform independent, are the basis on which several 
new transformations, one for each target platform (see e.g. T2, T2’ and T3 in Fig. 1), 
can be defined. The output of these PIM-to-PSM transformations is the specification 
of our Web application for a given platform. At this level of abstraction, the models 
can still endure a final PSM-to-code transformation, usually implemented in WebSA 
by means of templates. In this way, the WebSA process guarantees the traceability 
from analysis to code. In order to complete the specification of this process, WebSA 
formalizes the three transformations (PIM-to-PIM, PIM-to-PSM y PSM-to-code) by 
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means of QVT (Query View Transformation) [6]. QVT defines a transformation 
language that is based on an extension of the MOF 2.0 metamodel and which allows 
to link models situated in different views and map them through the different life 
cycle phases. Also, QVT extends OCL for the specification of queries and filters over 
the models. The inclusion of an architectural view in this process has a preeminent 
role for the completion of the specification of the final Web application, and drives 
the refinement process from analysis to implementation. In the next section we will 
center on this architectural viewpoint and how it influences the refinement process. 
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Fig. 1. WebSA Web Development Process. 



3 Logical Architectural View for Web Applications 

The logical architectural view is responsible for the definition of the logical 
components (subsystem, modules and/or software components) that collaborate in the 
system, as well as the relationship among them. In WebSA this view is made up of 
three models, namely (1) the Subsystem Model (SM), which determines the 
conceptual subsystems that make up our application, (2) the Web Component 
Configuration Model (WCCM), which decomposes each subsystem in a set of 
abstract components related by abstract connectors and (3) the Web Component 
Integration Model (WC1M) which, as its name may suggest, performs an integration 
of views. 
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3.1 Subsystem Model 

Also known as structural design, the Subsystem Model determines which are the 
subsystems that make up our application. 

This model is made up of two main constructs: 

• Subsystem: element of coarsest granularity in any Web application 
architectural design. It defines a group of software components developed to 
support the functionality assigned to a given logical layer. Subsystems are 
depicted in WebSA by means of the UML package symbol. 

• Dependency Relationship: link element that reflects the use dependencies 
between subsystems. In WebSA it is depicted as a UML dependency arrow. 

This model provides the most abstract perspective of the logical architecture view, 
and the subsystems obtained during this phase will be later identified with each logic 
layer in the application. This separation on layers is essential to reduce the complexity 
of the system, as it is justified in the “layering approach” pattern presented in [2], In 
this model, and in order to ease its construction, WebSA proposes the use of any of 
the five distribution patterns defined in [7], 

The different layers identified for a given system are reflected in the Subsystem 
Model by means of a stereotype associated to the subsystem package symbol. WebSA 
defines nine subsystem stereotypes, namely: «user interfaces «server», «business 
logic», ((presentations «dialog controls «process controls ((business objects «data 
access» and ((physical data». Additionally, WebSA provides a set of restrictions that 
apply to the possible relationships between subsystems. 

Once the different subsystems have been identified, we must specify the contents of 
each subsystem, that is, the set of abstract components that configure them. Such 
contents are specified in the Web Component Configuration Model, which is 
introduced next. 



3.2 Web Component Configuration Model (WCCM) 

The second model of the logic Architecture view is the Web Component 
Configuration Model, which consists of a set of abstract components and connectors, 
particular to the Web domain. These abstract elements, also based on the Conceptual 
Architecture, are the result of a refinement process performed on each subsystem 
identified in the previous model. The main constructs of the Web Component 
Configuration Model are abstract components and abstract connectors. 

• An abstract component represents an abstraction of one or more software 
components with a shared functionality in the context of a Web application. An 
example of it is a Client Page, which represents any artifact that contains 
information and/or interaction code relevant for the user. Note how this kind of 
component does not necessarily map to a single physical page but reflects a 
general task that must be performed by the application, such as showing certain 
information to the user. Abstract components are depicted with a UML class 
symbol. 

• The abstract connector represents a dependency relationship between two 
abstract components in the system, and is depicted with a stereotyped UML 
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dependency relationship. This relationship may affect either the interface or the 
whole component. 

In order to construct a WCCM, the designer may use any of the architectural and 
design patterns. Such patterns provide powerful configurations that, applied to 
abstract components, not only provide a reuse mechanism but also contribute to a 
more efficient development process. 



3.3 Web Component Integration Model (WCIM) 

The last model defined as part of the logical architectural view is the Web Component 
Integration Model (WCIM). This view is also known as integration model , as it 
connects the functional and architectural views under a common set of concrete 
components, modules and connectors which will eventually make up the Web 
application. This model is defined during the WebSA Platform Independent Design 
phase, and still centers on design aspects (components, their interfaces and their 
relations). 

The WCIM includes three main constructs: concrete components, modules and 
concrete connectors. Concrete Components (CC) are the smallest unit in the context 
of the integration model. It represents a software component in a given application 
domain, and it is obtained as an instance of an abstract component. A module (M) is a 
container of one or more concrete elements in the context of a given Web application. 
Such elements can be either other modules, concrete components or concrete 
connectors. Modules condense the functionality of a set of elements, reducing in this 
way the complexity of the models, and are depicted by means of a UML package 
metaclass. Last, Concrete Connectors (CN) express a relationship between two 
concrete components or modules of the system. It can be regarded as an instance of a 
dependency relationship defined in the WCCM. The number of instances (concrete 
connectors) generated for each abstract connector depends both on the cardinality of 
such abstract connector and on the existing relationships between those abstract 
components and the functional side of the application. Concrete connectors belong, 
just as modules and concrete components, to a given application domain. 



4 Conclusions 

In this paper we have presented a Web development approach called WebSA. WebSA 
fosters the use of the MDA philosophy to define a set of suitable models and a 
complete refinement process to cover the Web application domain. In this way, 
WebSA adds a new architectural viewpoint to explicitly address the architectural 
issues. This view is made up of three models, and its construction follows a top-down 
process that goes from the Subsystem Model, where the layers of the application are 
defined, to the Web Component Integration Model, where the designer determines the 
low level platform-independent components that make up the final application. In 
each phase WebSA promotes the use of a set of reuse practices and provides the 
mechanisms to reflect different sets of requirements. Currently, we are working on a 
formal definition of the UML 2.0 profile and metamodel for WebSA, and also on the 
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set of QVT transformation models that support the WebSA refinement process, which 
we expect to be incorporating in the VisualWADE Development Enviroment [9] in 
the near future. 
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Abstract. Most of Web sites are built as a matter of priority. Therefore, to 
reduce the development time, the conceptualization phase is often put aside and 
the associated documentation is neglected. Moreover, during the exploitation 
phase, Web sites suffer the effects of a rapid and unstructured evolution 
process. Their reconstruction encompasses inevitably a reverse engineering 
process. In this paper, we propose RetroWeb, a reverse engineering approach of 
semi-structured Web sites. It aims to provide a description of the site 
informative content at the physical, logical and conceptual levels. This 
approach uses, at each level, a meta-model which is instantiated using reverse 
engineering rules. 



1 Introduction 

In spite the effort and the generated high costs for their development and 
maintenance, most enterprise Web sites are not suitable. The traditional principles of 
the information systems design and documentation are often neglected on behalf of 
the visual and esthetic aspects. The lack of conceptualization during the development 
leads to maintenance problems and to a bad structuring of the information over the 
Web site pages. To reconstruct already existing Web sites that do not respect the 
development life cycle, the reverse engineering is essential. It aims to extract, in a 
clear and formal way, at various abstraction levels, all available information in order 
to understand the site functionalities. 

This paper presents an approach, called RetroWeb, to reverse engineer the 
informative content of semi-structured Web sites. Besides the EER conceptual model, 
two meta-models are proposed to describe, at three abstraction levels, the semi- 
structured data coded on the Web site HTML pages. The first one is used to represent 
the Web site through its physical views. It is instantiated using the semi-structured 
data extracted from each HTML page. The second one is used to describe the Web 
site through its logical views. Mapping rules are proposed for the translation of 
physical views into logical views and then into conceptual ones. The whole site 
conceptual description is obtained by merging the generated conceptual views. 

The remainder of the paper is organized as follows. Section 2 describes RetroWeb. 
Section 3 discusses the related works. Section 4 concludes and presents future work. 
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2 RetroWeb Approach 

RetroWeb is an approach to reverse engineer the informative content of semi- 
structured Web sites 1 . It is built on the inversion of the life-cycle design process. 
Starting from web site HTML pages, it deduces an EER conceptual description of its 
informative content. It encompasses three steps: the extraction, the conceptualization 
and the integration steps. The following paragraphs describe each of these steps. 



The Extraction Step. It aims to the retrieval of the semi-structured data coded on the 
site HTML pages and to describe them through physical views (one physical view per 
page). Its result is the instantiation of the meta-model describing the extracted 
physical views. For instance, let us consider a Web site for academic journal 
publications. The left part of Fig. 1 presents a Web page that displays, for each 
volume of the academic journal, its authors. The right part of the same figure gives its 
corresponding physical view. 




Fig. 1 . An academic journal publications Web site page and its corresponding physical view 

The concepts used to build physical views are: simple variable, simple variable 
domain, composed type and multi-valued type variables. A simple variable is an 
atomic structure that can hold an atomic piece of data. Simple variable domain is the 
values set (data) that can be hold by a simple variable in a page. A composed type 
variable is a data record build by one or more simple variables. A multi-valued 
variable is a set of composed type variables. 

The extraction step is performed into three phases: pre-processing, extraction and 
naming phases. The pre-processing phase takes as an entry HTML pages, corrects 
them, proceed to some cleaning, executes some transformations and then returns, for 
each page, a coded sequence describing the page structure. In this sequence, the 
structure tags are codified using the same number of positions. All textual data that 
are not identified as tags are replaced by a token "Text". The second phase deduces 
pattern expressions that will be used by the wrapper to extract data from pages. It uses 
the DeLa system technique described in [1]. The last phase assigns significant names 



1 Semi-structured web sites are in majority data-rich and display data in contiguous blocks. 
These blocks are ordered and aligned such that they exhibit regularity. 
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to variables of the physical views. It uses an algorithm that improves the labeling 
itself by reducing the number of concepts to name. It first defines classes of concepts 
that may be assigned the same label. Then, it assigns to any concept, not yet labeled, a 
name and gives this name to its family (i. e. to all concepts sharing the same class). It 
uses, to that end, some heuristics. More details about this phase can be found in [2], 



The Conceptualization Step. It aims to produce the EER schemas associated with 
the physical views. In order to reach this result, the conceptualization step translates 
first the physical views into logical ones, constructs for each logical view its 
corresponding EER schema and then, affects, according to the same naming process 
used in the precedent step, significant labels to entity-types and relationship-types of 
the obtained conceptual schemas. Consequently, it generates successively an instance 
of the meta-model describing the logical views of the web site pages and an instance 
of an EER model. For our example, the physical view described in Fig. 1 is 
transformed into the logical view of Fig. 2a and then into the EER schema of Fig. 2b. 




Fig. 2. The logical and conceptual views deduced from the physical view of Fig. 1 

A logical view is described through three concepts: the property, the view and the 
multi-view concepts. A property is a logical representation of a variable. A view is a 
logical representation of a composed type variable. It groups properties that describe 
an object represented in a web page. It can have one or several occurrences. A multi- 
view is obtained by assembling all deduced views. It can also have several 
occurrences. 

The different schema transformations are performed thanks to reverse engineering 
rules. For example, among rules used to translate logical views into conceptual ones, 
we can quote: 

- Rule 1 : every logical view becomes an EER schema 

- Rule 2: each view of a multi-view becomes an entity-type of the EER schema 

- Rule 3: if two views VI and V2 belong to the same multi- view then the entities 
that they represent are linked by a relationship-type. If the two views have a 
number of instances higher than 1 then the cardinality of this relationship-type is 
M: N. In contrary, if one of the two views has a number of instances equal to 1, 
then the cardinality of this relationship-type is 1 : N. 
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The Integration Step. It merges the portions of EER schemas into a global one in 
order to give a global conceptual description of the whole web site informative 
content. This step is based on integration techniques well known in the information 
systems context. The majority of these techniques propose an integration in 4 phases: 
(i) a pre-integration phase which aims to standardize the schema sources by 
translating them into a common conceptual model, (ii) a comparison phase whose aim 
is to identify the relations between schema sources, (iii) a fusion phase which allows 
the merging of the schema sources in an integrated one according to the results of the 
precedent phase and to the existing integration rules and finally (iv) a reorganization 
phase that improves the quality of the integrated schema. We re-use these phases. We 
choose the EER model as a common conceptual model. 



3 Related Works 

The evolution of Web sites has been addressed from various ways. Some research 
works take an interest to the evolution of the presentation [3, 4] and others to the 
restructuring of the HTML code [5, 6, 7]. [2] proposes an approach that allows small 
display units, like PDA and WAP, to access to the Web site content. [5] describes a 
clustering technique to translate static pages into dynamic ones. [6] uses a slicing 
technique to reduce the site size, [7] applies re-writing rules to the HTML code. 

The literature also supplies approaches that aim to obtain Web sites abstract 
representation [8, 9, 10, 11, 12]. [8] presents a framework to deduce, from XML 
pages, the corresponding DTD. [9] analyzes web site code in order to automatically 
reconstruct the underlying logical interaction design. [10] translates the visual layout 
of HTML forms into a semantic model. [11] produces HTML Uls by integrating data 
of several web pages. [12] uses UML diagrams to model views of web applications, at 
different abstraction levels. 

Other related works concern data extraction from HTML code. Their principal 
concern is the retrieval of data concealed in semi-structured data-rich pages. The way 
in which these data will be displayed, and the models to which they will be mapped, 
are left to the user. 

Through RetroWeb, we wish to recover the informative content of the whole site. 
Thanks to the proposed meta-models, RetroWeb supplies, at physical, logical and 
conceptual levels, a clear and semi-formal description of the web site informative 
content. This extracted description is useful for its re-documentation, re-structuring or 
integration with other Web sites. 



4 Conclusion 

In this paper we have proposed a reverse engineering approach of semi-structured and 
undocumented Web sites, called RetroWeb. RetroWeb gives a description of the 
informative content of the site at various abstraction levels: physical, logical and 
conceptual levels. Reverse engineering rules are defined to map the physical 
description into the logical one and then into the conceptual one. 
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According to the type of needed evolution, the web site maintainer can execute 
totaly or partially the reverse engineering process. For instance, RetroWeb can be 
used either for the integration of Web sites or for the translation of FITML sites into 
XML sites. In the firts case, the conceptual description of the informative content of 
the two sites must be retreived. In the second case, the retrieval of physical views can 
be enough. 

Our current work involves implementing RetroWeb. Further works will mainly 
concern the enrichment of the set of heuristics used for the naming of concepts. We 
also expect to enrich the reverse engineering rules set in order to exhibit, for example, 
generalization-specialization links at the conceptual level. Finally, we wish to extend 
our process to other aspects like the site navigational structure. 



References 

1. Wang, J., Lochovsky, F.: Data extraction and label assignment for Web databases. Proc. of 
the 12th International Conference on World Wide Web, Hungary (2003 ) 187-196 

2. Essanaa, S., Lammari, N.: Improving the Naming Process for Web Site Reverse 
Engineering. Proceedings of the 9th International Conference on Application of Natural 
Language to Information Systems, Manchester June (2004) 

3. Vanderdonckt, J., Bouillon, L., Souchon. N.: Flexible Reverse Engineering of Web Pages 
with Vaquista. Proceedings of the 8th Working Conference on Reverse Engineering 
(WCRE'01), October (2001) 241-248 

4. Lopez, J. F., Szekely, P.: Web page adaptation for universal access. In Stephanidis, C. (ed.) 
Universal Access in HCI: Towards an Information Society for All. In proceedings of the 
1st International Conference on Universal Access in Human-Computer Interaction, New 
Orleans August (2001). Mahwah, N. J.: Lawrence Erlbaum Associates 690-694 

5. Ricca, F.. Tonella, P.: Using Clustering to Support the Migration from Static to Dynamic 
Web Pages. Proceedings of the 1 1th International Workshop on Program Comprehension, 
Portland Oregon USA May (2003) 207-216 

6. Ricca, F., Tonella, P.: Construction of the System Dependence Graph for Web 
Application Slicing. Proceedings of SCAM'2002, Workshop on Source Code Analysis and 
Manipulation, Montreal Canada, October (2002) 123-132 

7. Ricca, F., Tonella, P., Baxter, I.. D.: Web Application Transformations based on Rewrite 
Rules. Information and Software Technology Volume 44(13) (2002) 811-825 

8. Chuang-Hue, M., Ee-Peng, L., Wee-Keong, N.: Re-engineering from Web Documents. 
Proceedings of the International Conference on digital Libraries (2000) 148-157 

9. Paganelli, L., Paterno, F.: Automatic Reconstruction of the Underlying Interaction Design 
of Web Applications. Proceedings of the 14th International Conference on Software 
Engineering and Knowledge Engineering (SEKE 02), Ishia Italy July (2002) 

10. Gaeremynck, Y., Bergman, L. D.. Lau, T.: MORE for less: model recovery from visual 
interfaces for multi-device application design. Proc. of the Int. Conf. on Intelligent user 
interfaces, Miami Florida USA January (2003), ACM Press, New York USA (2003) 69-76 

11. Stroulia, E., Thomson, J., Situ, Q.: Constructing XML-speaking Wrappers for Web 
Applications: Towards an Interoperating Web. Proc. of the 7th Working Conference on 
Reverse Engineering (WCRE’2000), Queensland Australia (2000), IEEE Computer 
Society 

12. Di Lucca, G. A., Fasolino, A. R., Pace, F., Tramontana, P., De Carlini, U.: WARE: a tool 
for the Reverse Engineering of Web Applications. Proc. of the European Conference on 
Software Maintenance and Reengineering (CSMR2002), Budapest March (2002) 




Empirical Methodologies for Web Engineering 



Briony J. Oates, Gary Griffiths, Mike Lockyer, and Barry Hebbron 



School of Computing 
University of Teesside 
Middlesbrough 
TS1 3BA, UK 

B . J . Oates@tees .ac.uk 



Abstract. We review a range of data generation methods and empirical 
research strategies of potential usefulness to web engineering research. The 
various strategies do not all share the same underlying philosophy about 
knowledge and how it can be acquired. We therefore explain two contrasting 
philosophical paradigms: positivism and interpretivism. We suggest that 
empirical web engineering should use a plurality of research strategies and data 
generation methods, and recognise the potential usefulness of both positivism 
and interpretivism. Finally we discuss the implications of such a plurality. 



1 Introduction 

The majority of web engineering research, like software engineering research, 
concentrates on design research i.e, developing new concepts, models, methods or 
instantiations [1 ]. However, there have been strong criticisms of software engineering 
researchers for under-usage of empirical studies and failing to validate their research 
ideas [e.g. 2], There is therefore increasing interest in empirical software engineering. 
If web engineers are to avoid similar criticisms, they must be able to both perform 
empirical studies and also assess the empirical research findings of others. We 
therefore review a range of empirical strategies and data generation methods of 
potential usefulness to web engineering research and summarize two contrasting 
philosophical paradigms: positivism and interpretivism. We argue that both 
philosophies and all the strategies and methods are relevant to web engineering 
research. 



2 Data Generation Methods and Research Strategies 

Data and data analysis can be either quantitative (i.e. numeric), or qualitative (i.e. 
textual, verbal or visual). Data generation methods available for gathering evidence 
include questionnaires, interviews, observations and documents (which include 
multimedia ‘documents’: non-textual artifacts such as photographs, videos and 
screenshots). Research strategies are the ways in which data generation methods are 
used and combined. One strategy can use many data generation methods, although 
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particular strategies may be associated with particular methods, and typically one 
research strategy addresses one research question. 

Strategies for using and combining data generation methods include experiments, 
surveys, case studies, action research and ethnography. Table 1 below summarizes 
each strategy, provides references where more information about each strategy can be 
found, and gives examples of its use in web-related research. 



Table 1 . Summary of research strategies for web engineering 



Strategy 


Brief description 


Examples 


Experiments 


Use observations to look for evidence of 
cause and effect, so can confirm or refute a 
hypothesis [3] 


[4,5] 


Surveys 


Systematic gathering of information from a 
large sample, looking for general trends or 
patterns via statistical analysis [6] 


[7-9] 


Case studies 


Rich account of particular experience, event 
or situation, often longitudinal view [10, 11] 


[12, 13] 


Action research 


Developers research iteratively into own 
practice, with twin aims of contributing to 
practical concerns of people in a situation 
and to the goals of science [14] 


[15, 16] 


Ethnography 


Researchers immerse themselves in lives of 
the people under study, experience the same 
as them, and place phenomena they observe 
in their social and cultural context [17] 


[18, 19] 



3 Philosophical Paradigms 

The strategies in Table 1 are based on different philosophical assumptions about the 
nature of ‘reality’ (i.e. ontological assumptions) and about the nature of ‘knowledge’ 
and how it can be obtained (i.e. epistemology). These are summarized in Table 2. 

Positivism underlies the scientific method, which has been developed by the 
natural sciences [e.g. 20]. Many people know of only this approach to research, and 
our modern daily discourse is frequently based, often unthinkingly, on a positivist 
worldview, with politicians and journalists demanding ‘proof’ and ‘the truth’. 
Interpretivism [eg. 21] has been developed by the social sciences, and recognises that 
the social world has few equivalents to the ‘laws of nature’ in the physical world. For 
example, there is no guarantee that two people joining in with the life of a web 
development department as ethnographers would gather the same data and interpret it 
in the same way to draw the same conclusions. 
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Table 2. Summary of positivism and interpretivism 





Positivism 


Interpretivism 


Strategies 


Experiments and surveys 


Ethnography, most action 
research and many case studies 


Ontological 

assumptions 


Physical and social world 
exists independently of 
humans; exists ‘out there’ 
to be studied, captured and 
measured. Researcher 
‘discovers’ this world by 
measurement, modeling 
and observations. 


Whatever ‘reality’ is, it can only 
be accessed through social 
constructions such as language 
and shared meanings. 


Aims 


Generalizations - 
irrefutable objective facts 
and fundamental laws 


Understanding, how people 
make sense of their perceived 
worlds, and how those 
perceptions change over time 
and differ from one person or 
group to another 


Researcher 


Must be neutral, objective 
detached 


Can never be neutral: their 
assumptions, beliefs, actions 
inevitably shape research 
process and affect situation. 


Epistemology 


Empirical testability of 
hypotheses and theories, 
leading to verification or 
refutation, and a search for 
universal laws or 
principles 


Studying people and practices in 
their natural social or work 
settings 


Evaluation 

criteria 


Internal and external 
validity, reliability and 
replication 


Plausibility and cogency of the 
reasoning and the evidential 
data 



4 The Need for Plurality 

Our literature searches have confirmed the findings of Bahli and Di Tullio [22]: most 
empirical web engineering research has used surveys or experiments and a positivist 
perspective. In Section 2 we had to suggest web research examples for some 
strategies from the information systems, social sciences and education disciplines. 

Members of the web engineering community could decide that only positivist 
research is appropriate to their discipline. Or they could decide that an interpretive 
case study is only appropriate as an exploratory method of investigation prior to a 
more ‘scientific’ approach. We suggest, however, that web engineering should accept 
both positivism and interpretivism and recognise a wide range of research strategies 
[cf. 23]. Web engineering is dependent on the people and the environment in which it 
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is practised, making it difficult or impossible to design and implement carefully 
controlled and repeatable experiments. Where such experiments can be achieved, a 
positivist approach could provide some truths on which to build the discipline. Where 
this is not possible, interpretivist approaches such as ethnographies and case studies 
can help us to explore particular situations and contexts, in order to understand better 
how people understand, engineer and use web-based artifacts in the real world. Rich 
and detailed understanding from a series of case studies of web engineering might, 
but not necessarily, gradually accumulate into a generally applicable body of 
knowledge. 



5 Implications 

We are not proposing that everyone has to abandon positivism and adopt 
interpretivism. But our argument for plurality does mean that researchers and 
reviewers should not automatically reject a as ‘unscientific’ work which does not fit 
the positivist paradigm. On the other hand, researchers and reviewers do not 
automatically have to accept qualitative, interpretive evidence. They should know 
enough about the tenets of interpretivism to accept or reject the validity of such 
evidence on its own terms. 

Web engineering practitioners may also need educating in the different types of 
data generation methods and strategies and underlying philosophies of empirical 
research, especially if they are to be persuaded to adopt new practices on the basis of 
qualitative, interpretive evidence. Their own educational background might have only 
introduced them to the positivist scientific method, and, as we noted earlier, our 
modern daily discourse is frequently unthinkingly based on a positivist worldview. 

By using a wide range of strategies and data generation methods, and both 
positivist and interpretivist approaches, empirical web engineering research can 
increase our knowledge and understanding of how to develop, deploy and maintain 
high quality Web-based systems and applications. 
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Abstract. In tackling the Semantic Web, a rich domain that requires special 
attention is the Geospatial Semantic Web. A contribution to the Geospatial 
semantic Web is the definition of an approach for integrating non-spatial 
resources like HTML, PDF, GIF, etc., with spatial XML resources represented 
by Spatial XMF [4]. However, in order to put this approach into practice it is 
necessary to solve the problem of developing an integration system for querying 
spatial XML resources stored in different sources. In this paper, we have 
implemented an approach for querying spatial and non-spatial information 
represented in the Geographical Markup Language (GML). The approach uses 
RDF to integrate the spatial XML documents. A performance study has been 
carried out and the results are given. 



1 Introduction 

A rich domain that requires special attention is the Geospatial Semantic Web [1]. The 
enormous variety of encoding of geospatial semantics makes it particularly 
challenging to process requests for geospatial information. In the future, the 
Geospatial Semantic Web will allow the returning of both spatial and non-spatial 
resources to simple queries, using a browser. For example, a query “lakes in Maine” 
should return all relational resources with lakes in Maine (pictures, text, ...) in 
different formats (XML, HTML, JPG, PDF, References, ...) [1], 

However, in the same way as with the Semantic Web, in order to approach the 
Semantic Geospatial Web it is necessary to solve several problems. One of these is 
the integration of spatial and non-spatial resources with different schemas stored in 
different sources. 

Tackling the integration of spatial information on the Web is not a simple task 
since there are several High level problems (e.g. heterogeneity for representing spatial 
information, for using spatial operators, for defining the semantic of the objects, etc. 
[2]) and Low level problems (e.g. the sources may store large amounts of incomplete 
spatial data, which may make it necessary to join the results of queries with spatial 
joins [3]). 

In order to contribute to the Geospatial Semantic Web, in this paper we study a 
prototype for querying spatial XML resources. The main task of this approach is to 
provide users with a unique interface for querying spatial XML resources with 
different schemas, independently of their actual organization and location. It provides 



N. Koch, P. Fratemali, and M. Wirsing (Eds.): ICWE 2004, LNCS 3140, pp. 316-329, 2004. 
© Springer-Verlag Berlin Heidelberg 2004 




Using RDF to Query Spatial XML 



317 



the infrastructure for formulating structured spatial queries by taking into 
consideration the conceptual representation of a specific domain in the form of an 
ontology. The resources are integrated using RDF. This work is based on [4], and we 
detail how to use RDF for integrating non-spatial resources (FITML, PDF, etc.) with 
spatial XML resources represented by GML. The most novel and critical feature of 
this approach is the querying of spatial XML resources, because it uses a different 
way from that of querying and relating non-spatial resources. Therefore, our prototype 
is focused on this alone. After proving the viability of this prototype, making a 
prototype in accordance with [4] is a simply task. 

In our study the spatial information is represented by GML because it is an XML 
encoding for the transport and storage of spatial/geographic information, including 
both spatial features and non-spatial features. The mechanisms and syntax that GML 
uses to encode spatial information in XML are defined in the specification of 
OpenGIS [5], Thus, GML allows a more homogeneous and flexible representation of 
the spatial information. 

Query mediation has been extensively studied in the literature. Directly concerned 
with the integration of XML resources, it is worth noting C-Web Portal [6] and [7]. C- 
Web Portal supports the integration of non-spatial resources on the Web, and it 
provides the infrastructure for formulating structured queries by taking into 
consideration the conceptual representation of a specific domain in the form of an 
ontology. On the other hand, [7] proposes a mediator architecture for the querying 
and integration of Web-accessible XML data resources (non-spatial data). Its 
contribution is the definition of a simple but expressive mapping language, following 
a local as view approach and describing XML resources as local views of some global 
schema. Both of these approaches have aims in common with our approach, but they 
are only focused on non-spatial information. 

In relation to spatial XML integration, the approaches developed by [8], [9] and 
[10] stand out. [8] presents a mediation system that addresses the integration of GIS 
data and tools, following a global-as-view approach. It has a multi-tier client-server 
architecture based on WFS and uses standard wrappers to access data, extended by 
derived wrappers that capture additional query capabilities. [9] extends the MIX 
wrapper-mediator architecture for integrating information from spatial information 
systems and searchable databases of geo-referenced imagery. MIX is focused on 
integrating geo-referenced imagery but our approach is focused on spatial geometries. 
On the other hand, [ 10] designed a novel approach for integrating GML resources. 
The proposed architecture uses a Catalog expressed by RDF to relate the GML 
resources. Although [10] has the same aims as [4], it has a different focus. 

An overview of the architecture offered in this work is detailed in Section 2, which 
contains the most important features of our prototype. Section 3 provides some results 
from our performance studies. Section 4 contains conclusions and projected future 
work. 



2 Overview 



The main task of an integration mediator is to provide the users with a unique 
interface for querying the data, independently of its actual organization and location 
[11]. This interface, or global schema, is described as an ontology expressed with 
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RDF(S). As used here, an ontology denotes a light-weight conceptual model and not a 
hierarchy of terms or a hierarchy of concepts [7], The global schema can be viewed as 
a simple object-oriented data model. Hence, a global schema can be viewed as 
defining a database of objects, connected by roles, with the concept extents related by 
subset relationships as per the isA links in the schema. Since it is an integration 
schema, this is a virtual database. The actual materialization exists in the resources. 

Users pose queries in terms of global schema (RDFS). As a consequence, the 
mediator system must contain a module ( Solve mapping) that uses the resource 
descriptions in order to translate a user query into a query that refers directly to the 
schemas of the GML resources. For this purpose, we establish a correspondence 
between each resource and the global schema using RDF instances. In the Solve 
mapping module, an execution plan is not necessary because a query will be executed 
over a source if it fully satisfies all attributes of the query. 

In addition, this approach supplies descriptions of the resources and specifies 
constraints on resource contents (e.g. contains information about “Madrid”). These 
descriptions are stored as alphanumeric data in RDF instances (Filter Resources). 

A Wrapper receives a query over a schema of the resources from the mediator 
system. Here, the query is executed and the results of the query are expressed in GML 
format, which are then returned to the mediator. 

This process is a well-known pattern in applications over non-spatial XML 
documents. However, to apply this architecture efficiently over spatial XML 
documents it is necessary to take into account the following points: 

1 . The spatial query language used for the users should have the same semantic as the 
query language used in the sources, so the translation between both languages is 
easier. Because of this, we have used the same spatial language to query the 
ontology and the GML data model on the sources[4]. This query language was 
born as a spatial query language over spatial semi-structured XML documents. The 
data model and the algebra underlying the query language are defined in [12]. The 
query language has a familiar select-from-where syntax and is based on SQL. It 
includes a set of spatial operators (disjoint, touches, etc.), and includes traditional 
operators (=, >, <, ...) for non-spatial information. 

2. Unlike XML, it is very difficult to make efficient queries over GML because the 
spatial operator requires a spatial index that cannot be created directly over GML. 
Due to this an efficient method for storing GML documents is necessary. In [13] 
we studied three storage models for storing and retrieving GML documents over 
RDBMS. This work concluded that the LegoDB approach [14] obtains the best 
performance. We have therefore used this approach in our implementation. 

3. Obviously, in schema integration the mediator can know when a resource has 
schema to satisfy a query. However, only when the query is executed in the 
resources does the mediator know whether the information stored in the resource 
satisfies the where clause. Thus, for example, it is possible that 100 resources have 
schema to satisfy the query, but only 10 have the information that the user wants. 
In this case, posting the query to each resource (100) is inefficient. To solve this, 
our approach uses the above-mentioned description to obtain a priori a set of 
candidate resources (in the same way as a Catalog in [15]). 

In the following section the most important features of our prototype are detailed. 
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2.1 Ontology 

The solution proposed in this paper is based on [4], which details how to use RDF(S) 
for querying spatial XML resources (GML). [4] is based on Community Web Portcil 
(C-Web) [16]. C-Web essentially provides the means to select, classify and access, in 
a semantically meaningful and ubiquitous way, various information resources for 
diverse target audiences. However, in [4] we do not only use the RDF(S) to relate 
resources (html, pdf, jpg,...) situated in different web sites; we also use it to know 
how a schema satisfies an ontology. These schemas are always expressed as DTDs for 
each source. Each site can offer the schemas that satisfy, fully or partially, the 
ontology defined for an interest community. Therefore, unlike the original C-Web, it 
is possible to apply spatial operators (comparatives: cross, overlap, touch; analysis: 
Area, Length) over the resources provided that they represent geometry information 
with GML. 

In Figure lan example of the Portal Schema and its instances is shown. Due to 
RDF’s capability for adding new feature and geometry types in a clear and formal 
manner, this example has been carried out extending the geospatial ontology defined 
by OpenGIS [5], where the class ( Geometry , LineString, etc) and properties 
(i coordinates , PolygonMember, etc) are defined. The example shows a simplification 
of a City Model. In this, CityModel contains Blocks and a Block contains Parcels, and 
a CityModel has the properties name, population and Category. 

The most important feature of this approach is that there are two kinds of 
properties: (1) properties that describe the data offered by the resources 
(descriptions), represented by a rectangle filled with white; (2) properties that 
represent the name of the attributes of a schema (DTD) that have the same semantic, 
represented by a rectangle filled with gray. 

The first set of properties such as Name and Category in CityModel or Type in 
Parcel make the Catalog. It allows a spatial query to be refined before it is posted to 
the sources. For example, if we wanted to execute a query about parcels in 
“Albacete”, it would not be necessary to execute the query in all the resources with 
information about parcels, but only in those resources related to the CityModel called 
“Albacete”. At the beginning, users filter resources in accordance with their 
description in order to get the candidate resources, and then users can apply a spatial 
query over these alone. 

Evidently, a large Catalog allows better filtering of a query, but it leads to difficult 
management. Owing to this, the size of the Catalog should be defined by consensus 
for each problem. 

With regard to the second set of properties, in our prototype the attributes are 
referenced by beginning with the root of the XML documents. For instance, the 
resource &r2 (Block) located in www.cities.com/Blocks.gml represents in a DTD the 
“ Boundary ” property with the attribute “ world.city.BoundaryCity ”, and the same 
property is represented by &r3 with '' Block. Boundary By ' . Figure 2 shows an example 
of a data model for the resource &r3. This data model is obtained to start from a GML 
document [12]. Note that dot notation is used to describe the attributes in the spatial 
XML data model. It is the same syntax used in our query language mentioned above. 
Thanks to this, a translation between notations (query and mapping) is not necessary. 
In addition, wildcards and other features related to semi-structures can be used to 
define attributes [12], 
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Portal Schema 




rl: wyvw.alba.es/cMes.gml 
r2: i n n r. cilL’s. com Blocks. %ml 
r3 : www. example, comManzaa. gnl 
r4: www. cede. es AlbaParcste ipnl 






'310<H P<CC«l.EXLCVl" 



Fig. 1. Portion of Catalog of a CityModel. 



On the other hand, spatial operators defined with the syntax of our query language 
can also be used to enrich the meaning of a property in a resource. Thus, for example, 
the property Area in the concept Parcel is defined in &r3 as a function 
‘AREA[Block.BoundaryBy]’\ In this case, &r3 does not have an attribute to describe 
the property Area , but it is possible to obtain data with the same meaning applying an 
operator Area[] over Block.BoundaryBy. In this way, operators like Length, Area, 
Buffer, ConvexHull, Union, Intersection, etc. can be used. This feature can only be 
used when it is applied over attributes in the same resource. 

With the Portal Schema and the Portal Resources Description we have all the 
information necessary to know what resources there are and how to satisfy a query. 
For example, we define the following query over the ontology shown in Figure 1: 
‘'Obtain the Area of a Block where the Parcel(extendof) touches Block(boundary) and 
the Parcel(Number) > 20”. This query is represented in our query language over a 
data model query language in Table 1(a). 

According to Figure 1 the only resource that can satisfy all the properties defined in 
the query is &r3. Table 1(b) shows the results of applying translation of the properties 
in the ontology to real attributes in the resource &r3 (Figure 2). Note that it is a very 
simple translation because we use the same query language and we do not have to 
adapt the semantic between them. Note that the relation between Parcel and Block in 
the Resource &r3 is not included in the query. This relation is established on the 
Wrappers. 
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Table 1. Example of translation 



SELECT Block.Area 
FROM Block contains Parcel 
WHERE Parce.Number > 20 AND Block. extendof_B 
TOUCHES Parcel. extendOf 


SELECT AREA[ B. boundaryBy ] 

FROM Block AS B , Block.Parcel AS P 

WHERE P.Number>20 AND B.extendof_B TOUCHES 

P. extendOf 


(a) 


(b) 




Fig. 2. Example of Data model - Resource &r3 

Note that the queries are executed firstly over the catalog to filter the resources that 
have useful descriptions for the user. Finally, the user makes a spatial query over 
these resources. However, the description of the user interface for carrying out these 
queries is not the aim of this paper. 



2.2 Wrapper 

Generally speaking, a wrapper should execute the query and generate the results as a 
GML document. However, as is mentioned above, the query over GML in each 
source must be efficient. The spatial operator requires a spatial index that cannot be 
created directly over a spatial XML document. Thus, an efficient method for storing 
GML documents is necessary. In [13] we studied the behavior of different alternatives 
over XML documents (non-spatial data) applied to GML documents (spatial and non- 
spatial data). As a result, we selected a RDBMS model called LegoDB[14] because it 
obtains the best results in the study. In Figure 3, an example over the data graph in 
Figure 2 is shown. It is the simplest mapping proposed, usually called inlining. 
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Fig. 3. Example of LegoDB approach 

In view of this, we have developed a wrapper architecture to convert our query 
language over our data model to spatial SQL queries. This approach is not dependent 
on a particular DBMS, thus Oracle, Informix, DB2 or any spatial RDBMS or 
ORDBMS may be used. This approach can be summarized in two components: T- 
Relational and T-DBMS. 

T-Relational transforms the query expressed in our query language (following our 
data model) to a query over a Relational model expressed by LegoDB. The output of 
this component is a SQL query with the syntax of our query language. This 
component has three important tasks: (1) conversion of attributes that contain 
wildcards (%,#)[ 12], To do this, we need to store a structure with all the possible 
paths of the data model for each document that is queried. This structure is obtained 
from the XML Schema (DTD) of the documents; (ii) transforming each attribute of 
the query expressed in a data model (e.g. block.parcel.extentof) in the real name in the 
relational schema (e.g. TParcel. extentof); (iii) the automatic generation of From 
clause in the final SQL query, according to relations involved in the query. The 
relations included in the From clause are obtained from the attributes used in the 
query. In order to carry out this transformation, a definition of a mapping for the 
relation of our data model and the relational model is necessary. This mapping 
(expressed in XML) relates the semantic relation between a path in our data model 
with a pair (relation, attributes) in the relational model. 

T-DBMS transforms the result of the T-Relational component to a SQL query in 
the syntax and semantic of a particular DBMS. This component depends on the 
DBMS used in the implementation. The complexity of this component depends on 
how to satisfy a DBMS with the OpenGIS specifications [13]. In this implementation 
we have used Oracle 9i, which follows most of the specifications defined in 
OpenGIS. 

In Table 2 a translation of a query in Table 1(b) to a SQL query is shown. This 
query has the syntax of Oracle 9i. Note that the relations between relations are 
included. 
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Table 2. SQL query 



SELECT 

MDSYS. SDOjGEOM. SDO_AREA(TBLOCKS.BO UNDARY,MDSYS. SDO_DIM_ARRA Y(MDSYS. SDO _DIM_ELEM 
ENT('X\ 0, 10000, 0), MDSYS. SDO_DIM_ELEMENT( 'Y', 0, 10000, 0))) 

FROM TBLOCKS, TPARCEL, TBLOCKMEMBER 

WHERE ( ( (TBLOCKMEMBER.P _BLOCK_lD = TBLOCKS. BLOCKJD AND TPARCEL. P_BLOCKMEMBER_ID 

= TBLOCKMEMBER. BLOCKMEMBERJD ) ) AND ( (TPARCEL.NUMBER > 20) AND 

MDSYS. SDO_RELATE(TBLOCKS.EXTENDOF, TPARCEL.EXTENDOF, 'mask=TOUCH querytype = 

JOIN')= 'TRUE'))) 



3 A Performance Study 

In this section, we describe the experimental studies that were conducted in order to 
assess the efficiency of querying a GML document with our approach. We focus on 
the effectiveness of this approach in terms of translation of queries over an ontology 
into queries supported by the wrappers, translation of these queries into spatial SQL 
queries, query processing and generation of GML documents from the results. All the 
experiments were conducted on an 2300Mhz PC with 512Mb RAM, 40Gb hard disk 
with operating system Windows XP professional. The RDBMS used was OracIe9i 
Spatial 9.0.1, which we selected because it allows the storage of spatial objects and 
the application of spatial operators over them. Oracle’s Spatial object-relational model 
was used (SDO_GEOMETRY object) [17] for simplicity. However, as mentioned 
above, Oracle’s relational model or another RDBMS may be used. All experiments 
were conducted on the same computer, using different databases to the mediator and 
the wrappers in the same Oracle DBMS. In this way, we obtained a greater system 
load. 

The data set used in these experiments respects the data set used in [13]. This data 
set represents a City model where a city has several blocks, each block has several 
parcels and each parcel has an owner. We used a data set with 7.1 Mb approx, and 
5000 rectangular parcels. In the mediator, we used three data sets: D1 (2.1Mb 
approx.) with 1250 resources, D2 (4.4Mb) with 2500 resources and D3 (8.4Mb 
approx.) with 5000 resources. 

For this test, on the DBMS the same relations shown in [13] were created. We used 
the simplest mapping proposed by [14]. The relations stored the data model shown in 
Figure 3. Indexes on relational tables were also properly built to improve query 
processing as follows: the attributes: TState(state_id), TCitymember(Citymember_id), 
TBlock(block_Id), TBlockmember(blockmember_id), TParcel (parcel_id), TArquitet 
(arquitect_id) are primary key. TCitymember(parent_state_id), TBlock(parent_ 
Citymember_id ), TBlockmemberf parent_block_id ), TParcel (parent _blockmember_ 
id), TArquitet (Parent _parece_id) are foreign key. The Spatial Attributes 
TState(BoundaryBy), TBlock(BoundaryBy,Extendof) and TParcel (extendof) have 
been indexed with R-Tree [ 17]. In order to carry out this mapping efficiently the RDF 
is stored in a RDBMS following [15]. 
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3.1 Elapsed Time 

In our study, we have focused our attention on the following aspects involved in 
assessing performance: 

1 . Translating queries over an ontology into queries supported by the wrappers has an 
elapsed time that should be studied [11], 

2. Translating queries from our data model to the relational model (spatial SQL) 
needs a mapping process. It depends on the path length between the entities in the 
GML document [13]. 

3. Since we are dealing with an XML query language, a very large number of joins is 
necessary. It is one of the more important performance problems in the relational 
implementation of these query languages [18]. In addition, storing and querying 
spatial data requires a larger amount of resources than storing and querying 
alphanumeric data [19]. This elapsed time was studied in [13] in isolation from a 
real system, but now we study how it works inside our system. 

4. Generating GML documents from the results. This process can be very expensive 
because a spatial query can return a very large amount of spatial objects. 

For these aspects we have studied the following elapsed times: T1 is associated to 
the translation of queries over an ontology into queries supported by the wrappers 
(parser and mapping). T2 is associated to translation of these queries into spatial SQL 
queries (parser, mapping, generation of from clause and translation to SQL). T3 is the 
elapsed time for the execution of SQL sentences, and T4 is the elapsed time for the 
generation of GML documents from the result set. 

Firstly, a set of queries is used to study the behavior of these approaches, involving 
only alphanumeric data (users may only wish to query alphanumeric data of a GML 
document). Secondly, a set of queries with spatial and alphanumeric operators is used. 
We define the complexity of a query according to the number of properties included 
in the query, the number of relations involved in the final spatial SQL and the number 
of joins in SQL. Table 4 shows the number of joins and relations involved in the final 
SQL query. All queries return 2500 objects approx. We have used the data set D2 in 
the mediator. In the execution of the query only one resource satisfies the query. 



Table 3. Number of joins and relations in each query 



Query 


N° Joins 


Relations 




Query 


N° Joins 


Relations 


01 


0 


1 




07 


5+7+7 


6 


Q2 


4 


5 




Q8 


5+7+7 


6 


03 


4+1 


5 




09 


3+7+7 


4 


Q4 


5+1+1 


6 




Q10 


3+1+ls 


4 


OS 


2+1 


3 




Qll 


5+1+ls 


6 


Q6 


5+1+1 


6 




Q12 


5+1+ls+l 


6 



The elapsed query times for these queries are shown in Figure 4. Q1-Q4 are 
queries with only alphanumeric operators. For Q1-Q2 additional joins are not 
included. They have a progressive number of joins and relations. Q3-Q4 are queries 
with more properties in Select and Where clause and 1 additional joins in Q3 and 1 
additional joins and 1 none equi-joins in Q4. 
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Q5-Q12 are queries with spatial operators, some of which have spatial and 
alphanumeric operators. Q5-Q6 are spatial queries with spatial operators (Area, 
Length and Buffer). The number of joins and relations is doubled in Q6. 

The elapsed query times for these queries are shown in Figure 4. The elapsed time 
for the first mapping (mapping mediator - Tl) depends on the number of resources 
and the number of properties involved in each query. However, the results are not 
very sensitive to the increment in the number of properties. The second mapping 
(mapping wrapper - T2) has a longer elapsed time than the first mapping. The most 
expensive function in this mapping is looking for all relations involved in the query 
and the relations between them to construct the from clause. However, the depth of 
our data model limits this elapsed time. Note that Q4 and Q6 have the highest elapsed 
time because they have the highest number of properties and involve entities that are 
highly-nested in the schema. The time of Execution (T3) of the SQL statement 
depends on the number of joins and the number of spatial operators involved. Note 
that this elapsed time increases with several joins and spatial operators. However, as 
is mentioned above, the number of joins can be limited by applying more specific 
heuristics [20] . Finally, the elapsed time to create GML documents is highest. This 
elapsed time increases with the adaptation of spatial objects to GML. This process 
was carried out following a process outside the DBMS. The process used the Oracle 
sdoapi to adapt Oracle Geometries to XML (we explore the record set to adapt the 
objects). In this way, we do not depend on a particular DBMS and simplify the 
translation to SQL. In addition, in this process a conversion between names of the 
relational attributes to GML attributes is carried out. 

16000 - 




Q1 Q2 Q3 Q4 Q5 Q6 

Fig. 4. Elapsed query time (Q1-Q6) 



Q7-Q12 use spatial operators which are more expensive in time than Q5-Q6. Q7 uses 
a spatial Union and has a high number of joins. Q8-Q9 use spatial operators in select 
clause (Union, Area ) and comparison with spatial operators ( Length ) in where 
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Fig. 5. Elapsed query time (Q7-Q12) 



clause. Q10-Q12 are the most complex queries with a high number of joins and 
spatial joins. Q12 has Union and comparison with spatial operators (Area) in where 
clause for the spatial joins. 

The elapsed query times are shown in Figure 5. Note that when there are complex 
spatial operators with a higher number of joins, execution time increases. Q7, Q8 and 
Q9 include, in the select clause, Union operators, combined with Area operator in Q8 
and Q9. The elapsed time of these queries is greater than the previous set. Q10-Q12 
use spatial joins ( Intersect ). Evidently, this kind of queries gets the worst elapsed 
time. In addition, the elapsed time to generate GML documents increases when spatial 
joins are involved. This is due to the process used to adapt the geometries mentioned 
above. 

In view of this, with complex spatial queries, applying a specific study (following 
on from [14]) for storing GML documents over relational databases is necessary. In 
order to obtain a good performance querying spatial XML documents, it is preferable 
to increase the size of relations, and to reduce the number of joins between relations. 
Lor each source a particular heuristic must be applied, depending on the kind of 
information stored on them. In this way, we can reduce the number of joins in each 
query. This will reduce the elapsed time to execute spatial SQL and the elapsed time 
to get the from clause in the wrapper mapping. In any case, T2 plus T3 is much lower 
than the elapsed time for querying directly over GML [13]. However, the high elapsed 
time to generate GML documents could be reduced if we used specific tools offered 
by the DBMS (SQL/XML[17]). (This comparative is not included for reasons of 
space). 
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Fig. 6. Scalability Test 



3.2 Scalability Test 

With regard to scalability, three studies can be carried out: (i) scalability over the 
spatial Data, (ii) scalability over the mapping in the wrapper, and (iii) scalability of 
the translation of the query languages in the mediator. The first study does not depend 
on either the data in the mediator or the data in the wrapper; it just depends on the 
DBMS. A study with these features was carried out in [13]. The second study is not 
relevant because the size of the data needed to translate the queries in the translator is 
very small. Consequently, we only deal with the third study. 

In order to do this, we used the Dl, D2 and D3 data sets detailed above. Over these 
data sets we translated two queries with few attributes (3 attributes, Ql) and queries 
with a lot of attributes (15, it is not included in table 2). Thus, we can study the 
influence between the mapping over lots of resources involved in the query and the 
number of attributes of the query. In our test we suppose all resources stored in the 
data set satisfy the query (so, 1250 resources satisfy the queries in Dl, 2500 in D2 and 
5000 in D3). 

Figure 6 shows the elapsed time ratios for the two kinds of queries. The elapsed 
query times ratios are defined as follows. Assuming the elapsed time of a query using 
Dl is treated as scale t a , and the elapsed time of the same query is t b , using either D2 
or D3, the elapsed time ratio is t/t a . The study shows the difference in the number of 
attributes is not relevant with respect to the number of resources that satisfy the query. 
The elapsed time of the parse is inconsiderable. 

In conclusion, in our prototype the elapsed time of the mapping depends on the 
number of resources that satisfy the query and not on the number of attributes 
involved in the query. In the same way, as shown above, if there are 2500 resources 
but only one satisfies the query, the most significant elapsed time depends on the time 
to translate this query over this resource. 



4 Conclusions 

In this paper a prototype of a mediation system for querying XML spatial resources is 
studied. The main feature of this approach is the possibility of applying spatial 



328 



J.E. Corcoles and P. Gonzalez 



operators over the resources that are represented in GML format. We studied four 
elapsed times: regarding the elapsed time for the mapping in the mediator, we 
concluded that it depends on the number of resources involved in the query. For the 
same number of resources, the number of properties is not determinative. The elapsed 
time for the mapping in the wrapper is more expensive. Moreover, it is more variable 
because it depends on the heuristic used to represent a GML data model over the 
relational model in each resource. An heuristic with several relations offers a worse 
elapsed time in the mapping . The same problem can be applied over the execution of 
the query. The elapsed time for execution obtains better results with fewer relations. 
Finally, the worst elapsed time is obtained in the generation of GML. However, we 
can improve this elapsed time using specific tools offered by the DBMS. 

Future work foresees the development of a prototype to allow the splitting of a 
query between different sources, and the joining of the result in the mediator. The 
study presented in this paper will be used to check the efficiency of this future 
approach. In addition, we are developing a prototype to integrate GML resources and 
any other kind of resource (HTML, PDF, etc). Thus, with this approach it is possible 
to discover spatial and non-spatial resources which are interrelated semantically on 
the Web. In this way, this work represents a small step towards the Semantic 
Geospatial Web. 
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Abstract. In the semantic web environment it is important to be able 
to specify access control requirements about subjects accessing the infor- 
mation and about resources to be accessed in terms of the rich ontology- 
based metadata describing them. In this paper, we outline how current 
standard policy languages such as XACML can be extended to address 
this issue. Then, we describe a reference architecture for enforcing our 
semantics-aware policies. 



1 Introduction 

Current web-based applications describe resources (including users) and ser- 
vices via a number of XML-based standard protocols [10]. Among those, XML- 
based standard definitions for policies and credentials have recently received an 
increased interest [5,13]. Semantic web technologies are changing this picture, 
using advanced knowledge representation techniques for enriching e-business en- 
vironments. The semantic web is built around the notion of representing shared 
knowledge via standard ontologies, that are used by intelligent agents to under- 
stand the nature of the information they are processing [6]. In an interoperable 
e-business architecture based on the semantic web vision, ontology-based domain 
models are therefore used as controlled vocabularies for resources description, 
allowing users to obtain the right resources at the right time [3] . While research 
on developing standards and tools that ultimately will lead to the existence of 
the semantic web is increasing [16], security impacts of these new technologies 
have not been addressed sufficiently. On the semantic web, writing access control 
policies where subjects and objects are pointed at via data identifiers or simple 
predicates is not enough. Rather, it is important to be able to specify access 
control requirements about subjects and resources in terms of the rich metadata 
describing them. 

Some researchers have recently investigated security within the semantic web 
for the purpose of either expressing security policies or protecting semantically 
rich data. As an example of the two, [4,7] develop security ontologies that allow 
parties to share a vocabulary to exchange security-related information using a 
common language; while [9,12] present policy languages to specify access restric- 
tions over concepts defined in ontologies. Another line of work merging security 
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and semantic web concepts is presented in [14] as an approach for identifying 
Web inference channels due to ontology-based inference attacks. There, an on- 
tology is used to detect tags appearing in different XML documents that are 
ontologically equivalent (i.e., can be abstracted to the same concept in the on- 
tology), but which have contradictory security classifications. 

Although these approaches represent a first step toward the definition of a 
semantics-aware access control process, they do not completely exploit the power 
of the semantic web. Neither semantic web-based proposals nor emerging state- 
ful attribute-based security languages (e.g., XACML) allow support of access 
restrictions on resources according to complex semantics-aware assertions. 

In this paper we bring forward the idea of exploiting the semantic web to ex- 
tend current policy languages to allow the definition of access control rules based 
on generic assertions defined over concepts in the ontologies that control meta- 
data content and provide abstract subject domain concepts, respectively. These 
rules are then enforced on resources annotated with metadata regulated by the 
same ontologies. Our proposal allows for specifying access control requirements 
about: i) subjects accessing the information and ii) resources to be accessed in 
terms of rich ontology-based metadata associated with them. The result is a 
powerful policy language exploiting the high expressive power of ontology-based 
models. 

The contribution of this paper can be summarized as follows. First, we make 
the point for the need of more expressive policy languages and show how se- 
mantic web metadata formats can be exploited to this end. Second, we show 
how current standard security languages can be extended to include semantic- 
aware specifications. Third, we illustrate our architectural solution, including a 
semantic-aware authentication tool, called OntoPassport. Our proposal extends 
the extensible Access Control Markup Language (XACML) [5] by adding the 
capability to designate subjects and objects via generic RDF statements [15]. 
Our extensions to XACML introduce the following capabilities. 

— Semantic- aw are subject description. Semantic web applications need to eas- 
ily operate on subject descriptions to determine whether a policy rule is 
applicable to a user. 

— Canonical metadata syntax. The high expressive power of semantic web 
metadata allows for using different syntax to carry the same semantics. While 
no constraints can be posed apriori on the content of resources 1 descriptors, 
a standard syntax could be adopted for metadata used to describe subjects 
and objects within access control policies. Also, a standard syntax could be 
used for subjects’ descriptions. 

2 Basic Concepts 

In this section, we briefly overview the main standards on which our proposal 
is based, namely, XACML, SAML (the XML standard for encapsulating access 
requests) and RDF, highlighting the features relevant to our research. 




332 



E. Damiani et al. 



XACML 

XACML is the result of a recent OASIS standardization effort proposing an 
XML-based language to express and interchange access control policies. XACML 
is designed to express authorization policies in XML against objects that are 
themselves identified in XML. The language can represent the functionalities 
of most policy representation mechanisms. An XACML policy consists of a set 
of rules whose main components are: a target, an effect, and a condition . 1 The 
target defines the set of resources, subjects, and actions to which the rule is 
intended to apply. The effect of the rule can be permit or deny. The condition 
represents a boolean expression that may further refine the applicability of the 
rule. A request consists of attributes associated with the requesting subject, the 
resource involved in the request, the action being performed and the environ- 
ment. A response contains one of four decisions: permit, deny, not applicable 
(when no applicable policies or rules could be found), or indeterminate (when 
some errors occurred during the access control process). A request, a policy, and 
the corresponding response form the XACML Context. 

The subjects and resources to which a rule applies are defined by using a set of 
pre-defined functions (e.g., equality, set comparison, arithmetic) and datatypes 
(e.g., string, boolean, integer). To illustrate, consider the following portion of a 
rule. 



<Subject> <! — match role subject attribute — > 

<SubjectMatch 

Matchld= "urn:oasis:names:tc:xacml:1.0:function:string-equal"> <! — function — > 
<AttributeValue DataType= "http://www.w3.Org/2001/XMLSchema#string"> 
physician 

</AttributeValue> 

<SubjectAttributeDesignator Attributeld= 

"urn:oasis:names:tc:xacml:1.0:example:attribute:role" 

DataType= "http://www.w3.Org/2001/XMLSchema#string" /> 
</SubjectMatch> 

</Subject> 



This rule matches requests where the subject has an attribute role whose 
value is physician. While these functions and datatypes can be used to define 
many access control policies, XACML also specifies an extension mechanism for 
defining additional datatypes and functions. Our proposal makes use of such a 
mechanism to accommodate semantically rich constraints. 

SAML 

On corporate networks as well as on the global Net, access to services by single- 
sign-on authentication is becoming a widespread way to authenticate users. 

1 We keep at a simplified level the description of the language and refer the reader to 
the OASIS proposal [5] for the complete specification. 
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Single-sign-on authentication relying on digital certificates is currently the most 
popular choice for e-business applications. In this area, the most successful stan- 
dard is SAML [13], an authentication protocol handling authentication infor- 
mation across transactions between parties. SAML uses tagged sets of user at- 
tributes to represent subject-related information encapsulated inside service re- 
quests. One might wonder whether the semantics of user credentials could be 
represented directly by the standard XML schemata describing SAML. Unfor- 
tunately, standard XML schema definitions need to cover a wide repertoire of 
possible user attributes. For this reason, optional elements are widely used, thus 
decreasing the expressiveness of the schema as a descriptor of single instances. 
The goal of authentication standards is to carry information according to a pro- 
tocol, rather than describing a domain; also, a considerable number of SAML 
tags have a structural function and do not describe specific subjects. 

RDF 

RDF is a model to express generic assertions (RDF statements) associated with 
resources. An RDF statement is a triple of the form ( subject , predicate, object), 
where subject 2 is the resource being described, predicate is a property associated 
with the resource, and object is the value of the property. For instance, statement 
"index.html has been created by Lucy" can be represented by the triple 
(index.html, createddqy, Lucy). Due to the fact that the subject of a statement 
can be any resource, it is possible to make statements about statements in RDF. 
This technique is called reification. To clarify this important concept, consider 
the following statement: 

“Video http://www.acme.com/medical.avi shows Dr. Harry Morris assisting 
patient Carl Smith” 

To state this in RDF one must define several RDF statements: 

(Harry Morris, is-a, physician); 

(Carl Smith, is-a, patient) ; 

(Harry Morris, assists, Carl Smith); 

(http://www.acme.com/medical.avi, is-a, Video); 

(http://www.acme.com/medical.avi, shows, ‘‘Harry Morris, assists, Carl 
Smith’ ’ ) . 

Reification is represented by splitting a statement into three different parts 
(subject, predicate, and object) and asserting these parts. By reification, the 
previous sentences can be expressed as the following set of triples. 3 
(http://www.acme.com/medical.avi, type, Video) 

(Harry Morris, type, Physician) 

(Carl Smith, type, Patient) 

(assists, type, relation) 

(shows, type, relation) 

2 This is not to be confused with the use of subject in the authorization model. 

3 For the sake of clarity, we shall use an informal syntax where capital letters represent 
URIs and drop the use of XML namespace prefixes. 
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(A, type, statement) 

(A, subject, Harry Morris) 

(A, predicate, assists) 

(A, object, Carl Smith) 

(B, type, statement) 

(B, subject, http://www.acme.com/medical.avi) 

(B, predicate, shows) 

(B, object, A) . 

Reification is used often and, in our approach, a uniform RDF based on 
reification simplifies the evaluation procedure. 



3 Towards Semantic- Aware Access Control Language 

In this section we show how current XML-based standards can be extended to 
seamlessly incorporate metadata-based designations of subjects and objects. 



3.1 Including Assertion-Based Metadata in XACML 

The design of a policy evaluation and enforcement engine exploiting semantic 
web metadata needs to be based on a sound model and language for expressing 
authorizations in term of metadata. To this purpose, we chose to exploit the 
extensibility points already built in the XACML language rather than redesigning 
a policy language from scratch. Our extension points can be summarized as 
follows. 



— Extend the XACML Context to include metadata associated with both sub- 
jects and resources. 

— Extend the AttributeValue XACML element (used in XACML to qualify 
both subjects and objects) capability of specifying auxiliary namespaces . 4 
Auxiliary namespaces to be added are at least two: the rdf : one, allowing 
for using RDF assertions as values for the XACML AttributeValue element 
and another one (in our example, md : and ms : ) enabling using properties 
and class names from a user ontology within those assertions. 

— Extend the MatchJD attribute by introducing a new function, called 
metadataQuery, expressing the processing needed for policy enforcement. 

Although our proposed extensions to XACML rely on standard RDF syntax, 
some precautions should be taken to keep enforcement possible; namely, we 
prescribe that attribute values written in RDF use a RDF reification technique. 

4 Such additional attribute values are optional and do not disrupt parsability of stan- 
dard XACML policies using our extended schema. 
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Fig. 1. The Schema for a SAML assertion 
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Fig. 2. An example of resource and subject domain ontology 



3.2 Incapsulating Semantics- Aware Credentials in SAML 

Figure 1 shows the portion of SAML-XML Schema specifying the structure of 
an authentication assertion, the boxed element shows our proposed extension 
point. Basically, the SAML schema defines a subject identification and associate 
it with a set of attributes. The attribute definition is extremely open, leaving 
it to application-specific XML schemata to specify the actual set of attributes 
identifying the user. We extend the attributes allowed for the AttributeValue 
element to enable content including RDF assertions using suitable ontology con- 
cepts as predicate names. 

In the simplest case, the subject metadata can assert that the user holding 
the certificates belongs to a certain type such as (thisRequestUser , type, 
Physician), or more complex ones such as 

(thisRequestUser, type, Person) 

(thisRequestUser, buys , "Resource") 

(Resource, type, MovieDVD) 

(Resource, title, "Lord of the Rings") 

However, once again we use a canonical reified syntax. 
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< rdf : RDF 

xmlns:rdf="http: / /www.w3.org/TR/WD-rdf-syntax#" 
xmlns :md=" http: / / our domain, it /MD /Schema/ md-synt ax#" 
xmlns :ms=" http: / / our domain, it /MS /Schema/ ms-syntax#" > 

<rdf description 

rdf :about=" http: / / ourdomain.it/MD/Video/video010234.avi"> 

<rdf :type rdf :resource="http://ourdomain.it/MD/Schema/md-syntax# Video" /> 
<md: title > Treatment of Diseases</md:title> 

<md:duration> 1054067 </md:duration> 

< md : f ormat > a v i < /md : f ormat > 

< md : shows -how rdf:nodeID="content"/> 

</rdf : Description > 

<rdf description rdf :nodeID=" content" > 

< ms : surgeon > S am < /ms : surgeon > 

<ms : operates.on> Pat ient < /ms : operates.on> 

</rdf :Description> 

</rdf :RDF> 



Fig. 3. An example of RDF metadata associated with a video presentation 



3.3 Using the Extended Language 

To illustrate our examples of semantics-aware access control policies, we shall 
consider a medical digital library (MDL) that includes different kinds of multi- 
media data: free text, images, video, and audio. Each multimedia data item is 
complemented with descriptive metadata in the form of RDF descriptors. RDF 
descriptors could contain any assertion about the data items that can be writ- 
ten using the ontology vocabulary. However, in some controlled environments it 
might be possible to adopt the reification-based syntax greatly simplifying the 
evaluation procedure. In the following, we shall assume that the reified format 
of RDF statements is used. Note that however conversion tools are available 
capable to translate a variety of RDF syntax into the reified ones. 

To express the statements in our descriptors, we use three vocabularies. 
The first vocabulary contains standard terms such as predicate, statement, 
subject, object and standard relations such as is-a. This vocabulary is the 
RDFS base namespace (whose formal definitions can be found in [15]) containing 
the elements rdf : statement , rdf : subject , rdf : predicate , rdf : object , 
rdf : type that have a self-explanatory semantics. The second vocabulary, called 
resource domain ontology , contains domain-specific terms that are used to de- 
scribe the resource content such as Video and shows Jiow. The third vocabulary, 
called subject domain ontology, contains terms that are used to make asser- 
tions on subjects such as Physician, Patient, assists. Figure 2 illustrates 
the resource domain ontology and subject domain ontology where, for the sake 
of simplicity, we only report the main concepts that will be used to show the 
expressive power of our proposal. Note that generally speaking no assumption 
can be made about the format of RDF descriptors associated with resources. 
Figure 3 illustrates an example of RDF descriptor where, in addition to the clas- 
sical rdf : namespace, we use namespace md: for describing multimedia data and 
namespace ms : for describing medical staff. The RDF descriptor, associated with 
a video (video010234.avi), comprises of two sections: the first one expresses 
the document type according to a domain ontology and to some additional sim- 
ple metadata (e.g., format of the document, title); the second one contains more 
complex metadata expressing the fact that “Video video010234 . avi shows how 
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surgeon Sam operates on a patient”. Note that while the simple metadata in the 
first section could be expressed by a usual attribute-value pair, this is not the 
case for advanced metadata, to which our approach applies. Consider now the 
following protection requirement: 

Physicians of the Trauma Therapy Center department are allowed to see video 
presentations that show physicians assisting patients 

This requirement is composed of two assertions stating, respectively, 1 ) who can 
access the resource (Physicians of the Trauma Therapy Center department) and 
2) the kind of resources involved (Video presentations that show physicians as- 
sisting patients) . Such assertions are used to define the target of the XACML rule 
as illustrated in Figure 4. Consider now a request to see video video010234.avi 
submitted by a user who presents to our system subject metadata stating that 
the requester is Sam, a surgeon of the Trauma Therapy Center Department (see 
RDF description in Figure 5). Intuitively, by exploiting the hierarchical organi- 
zation of the concepts defined in the domain ontologies, the evaluation of this 
access request should return a permit decision because both Sam and the video 
involved in the request satisfy the subject and resource conditions specified in 
the rule, respectively. This is the result of the two following subsumptions: 

— Surgeon is a sub-class of Physician 

— assists is a sub-property of operates_on. 

We will see in more details the policy evaluation process in the next Section. 

4 Policy Evaluation 

When a policy involving metadata needs to be evaluated, the subject context 
already contains the RDF description of the requester, taken from the SAML 
request. Our policy evaluation engine works as follows. 

1. The semantic assertions about the requester that are included in the subject 
held of our policy rules and the metadata about the requester in the access 
request are compared to identify the policy rules that apply to the requester. 

2. The semantic assertions that are included in the resource context of applica- 
ble policy rules are used to query the descriptive metadata of the requested 
resource, to verify whether the requested resource satisfies the rules selected 
in the previous step. 

Both these selection steps involve RDF queries, where the assertions in the 
policy rules are used to query metadata associated with the requester and the in- 
volved resource. 5 A suitable query language is DQL, a logic-based query language 

5 Such querying can be tackled by means of two different techniques: reasoning based 
on metadata and database-like querying. The former approach considers RDF meta- 
data as a knowledge base that can be translated into logic programming clauses and 
applies reasoning techniques to them. 
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<?xml version*" 1.0" encoding="UTF-8"?> 

<Rule 

xmlns="urn : oasis : names : tc : xacml : 1 . 0 : policy" 
xmlns : xsi= http : //www . w3 . org/2001/XMLSchema- instance 
xmlns : ctx="urn : oasis : names : tc : xacml : 1 . 0 : context " 
xmlns : rdf ="http : //www . w3 . org/TR/WD-rdf-syntax#" 
xmlns :md=" http : / / oiirdomain.it/MD/Schema/md-syntax" 
xmlns :ms=" http : / / ourdomain.it/MS/Schema/ms-syntax" 

Ruleld="urn : oasis : names : tc : xacml : examples : ruleid : 1 " 

Effect="Permit" > 

< Target > 

<Subjects> 

<Subject> 

< Sub j ectMatch 

Matchld= "urn : ourdomain : function : metadataQuery " > 

< Attr ibuteValue 
DataType="http : //" > 

< rdf : Statement rdf :about="thisRequestUser" > 

< rdf : subject rdf : resource*" http:/ / ourdomain.it/MS/Schema/ ms-syntax# Physician" /> 
<rdf :predicate rdf :resource="http://ourdomain.it/MS/Schema/ms-syntax#belongs"/> 

< rdf : object rdf :datatype="http:/ /www. w3.org/ 2001 /XMLSchema#string" > 

Trauma Therapy Center 
< / rdf : ob j ect > 

</rdf : Statement > 

< / Attr ibuteValue > 

< Sub j ect Attr ibuteDesignator 
Attributeld="urn: ourdomain: attribute :metatag" 

DataType="http : / /www . w3 . org/2001/XMLSchema#string"/ > 

< /Subj ectMatch > 

< /Subj ect > 

</Subjects> 

< Resources > 

< Resource > 

<ResourceMatch 

Matchld*" urn: ourdomain: function: metadataQuery" > 

< Attr ibuteValue 
DataType="http : //" > 

< rdf Statement rdf : about*" thisRequestUrl" > 

< rdf : subj ect rdf : resource*" http:/ / our domain, it /M S/Schema/ md-synt ax# Video"/ > 

< rdf : predicate rdf : resource*" http:/ / our domain, it /MD/Schema/md-syntax#shows_how"/> 
< rdf : object rdf :nodeID=" content "/> 

</rdf : Statement > 

< rdf Statement rdf :nodeID=" content" > 

<rdf : subject rdf : resource*" http:/ / ourdomain.it/MS/Schema/ ms-syntax#Physician"/> 

< rdf : predicate rdf : resource*" http:/ /our domain. it /MS/Schema/ms-syntax#assists"/> 

< rdf : object rdf :datatype="http:/ / www. w3.org/ 2001 /XML Schema# string" > 

Patient 
< / rdf : ob j ect > 

< /rdf : Statement > 

< / Attr ibuteValue > 

<ResourceAttributeDesignator 
Attributeld="urn: ourdomain: attribute :metatag" 

DataType="http : //www . w3 . org/2001/XMLSchema#string"/ > 

< /ResourceMatch > 

< /Resource > 

< /Resources > 

< Actions > 

<Action> 

<AnyAction /> 

</Action> 

< /Act ions > 

< /Target > 

< /Rule > 



Fig. 4. An example of access control policy in extended XACML 

for the semantic web proposed in [2]. Here, however, we follow an SQL-like or 
an XQuery approach, assuming that RDF metadata about resources are stored 
as a relational or an XML database. 

First, let us examine the rule selection step. Suppose a request comes in 
whose encapsulated metadata are: 

(A, type, statement) 

(A, subject, thisRequestUser) 

(A, predicate, type) 

(A, object. Physician) 
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< rdf : RDF 

xmlns:rdf="http: / /www.w3.org/TR/WD-rdf-syntax#" 
xmlns :ms=" http: / / our domain, it /MS / Schema/ ms-syntax#" > 

<rdf : Statement 

rdf : about=" http: / / ourdomain.it/MS / medstaff/ 1 1234" > 

<rdf : subject rdf :resource="http://ourdomain.it/MS/Schema/ms-syntax#Surgeon" /> 
<rdf : predicate rdf : resource="http: / / our domain, it /MS/Schema/ ms-syntax# belongs" /> 
<rdf : object rdf :datatype="http:// www.w3.org/2001 /XMLSchema#string" > 

Trauma Therapy Center 
</rdf : object > 

< / rdf : Statement > 

<rdf : Statement 

rdf : about="http: / / ourdomain.it/MS / medstaff/ 1 1234" > 

<rdf : subject rdf :datatype=" http://www.w3.org/2001 /XMLSchema#rfc822Name" > 
http: //ourdomain.it/MS/Schema/ms-syntax#Surgeon" 

< / rdf : sub j ect > 

<rdf predicate rdf : resource="http: / / our domain, it /MS/Schema/ ms-syntax# name"/ > 
<rdf :object rdf :datatype=" http:/ /www.w3.org/2001/XMLSchema#string" > 

Sam 

</rdf : object > 

< /rdf : Statement > 

< /rdf : RDF > 



Fig. 5. An example of RDF description associated with a requester 



Then all XACML rules R whose subject metadata include (?, subject, 
Surgeon) will be selected. 

Let us assume that the resource metadata mentioned in the context of the 
policy rule R is the following: 

(?, type, Statement) 

(?, subject, Physician) 

(?, predicate, assists) 

(?, object, Patient) 

These metadata can now be used to build a query on the resource descriptors, 
to identify the objects to which the rule applies. For instance, the policy will 
apply to the video presentation with the metadata shown in Figure 3. 

The reified statement contained in the policy is used to construct the query 
which is submitted to the set of resource descriptors. Therefore, to evaluate the 
feasibility of our approach, the complexity of RDF query answering must be 
taken into account. RDF query evaluation process is composed of three main 
phases [8]: matches computation, minimization of queries, and redundancy elim- 
ination in answers. Here we shall only discuss the first contribution since some 
theoretical results are available from RDF querying research. The other two 
phases will need to be addressed with ad hoc solution for policy evaluation. 6 
In [8] the authors used the simpler problem of testing emptiness of the query 
answer as an approximation of the complexity of computing the matches. Distin- 
guishing between query complexity, considering the evaluation time as a function 
of query size for a given database, and data complexity, considering the evalua- 
tion time of a given query as a function of database size for a fixed query, they 
find out that evaluation is NP - complete for the query complexity and poly- 
nomial for the database complexity version. Furthermore, the size of the set of 
answers of a query q issued against a database D is bounded by |D| |qj , where \D\ 



Since query evaluation is often exponential in query size, static optimization of 
queries is an important goal. 
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Knowledge Hub 




Fig. 6. Our reference semantic web architecture 



is the size of the database (number of triples) and |g| is the number of symbols 
in the query. 



5 Our Architecture: The OntoPassport 



To present our architecture we will focus on our semantic web system, called 
Knowledge Hub (KH) [1], illustrated in Figure 6. In the KH, network resources 
are first downloaded in their native XML/XHTML format 7 , tidying them up 
if necessary. Then, a special-purpose tool, called OntoAssistant , is used to pro- 
duce RDF descriptions of their content. Such descriptions are written exploiting 
the standard vocabulary provided by a business ontology, written in the RDFS 
standard language. While more structured ontology description languages such 
as OWL [17] are now available, the KH setting is general enough for our cur- 
rent purposes. Users connect to the KH system via a Semantic Navigator that 
allows them to navigate/query metadata as well as the data themselves. Thanks 
to the body of knowledge comprising the ontologies and the metadata, the Se- 
mantic Navigator can act as a logic program, performing more sophisticated 
reasoning than a conventional query engine. Also, the Semantic Navigator can 
use the available metadata about resources and users themselves to customize 
their navigation experience and data presentation. To present a semantic web 
application like our KH with a semantics-aware description of a user, we can en- 
capsulate ontology-based metadata about the user within a SAML request. Our 



' Of course, it would be perfectly feasible to associate metadata with external resources 
without downloading them; however, this would bring up the problem of keeping data 
and metadata consistent, which is outside of the scope of this paper. 
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Fig. 7. The architecture of a security solution including the OntoPassport component 



OntoPassport tool was designed to handle this approach by encoding metadata 
in a X.509 certificate that can be carried by SAML. 8 

Figure 7 shows a closer view of our evaluation architecture. Figure 8 shows the 
WSDL interface definition for the OntoPassport implemented as a web service. 
The input datatypes correspond to the XML encoding of a generic certificate, 
while the return datatype is designed to cointain a set of RDF assertions that 
qualify the user with respect to a suitable role ontology. We could substitute the 
username-password authentication scheme with a traditional certificate. In this 
case the OntoPassport acts as a ’’translator” of an authentication certificate into 
a digital identity according to an ontology. 

The OntoPassport also includes in the certificate the name, URL and version 
of the RDFS Schema specifying the ontology it used to build the subject meta- 
data. This makes the type hierarchy used for subjects (and, more generally, the 
entire metadata vocabulary used to talk about them) easy to share and main- 
tain across an organization. We are currently working on the implementation of 
a PEP component including a fully featured RDF query engine. 

6 Conclusions 

Traditional access control models and languages result limiting for emerging 
Web applications. Although recent advancements allow the specifications of ac- 
cess control rules with reference to generic attributes/properties of the requestor 



Alternatively, our semantics-aware metadata can be saved in a secure cookie [11] on 
the user’s machine. 
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<?xml version* "1.0" encoding*" UTF-8"?> 

< def init ions name* " My WebS er vice " 
targetNamespace="http:/ /MyServer/OntoPassport.wsdl" 
xmlns="http: //schemas, xmlsoap.org/wsdl/" 
xmlns:xsd="http: / /www. w3.org/2001/XMLSchema" 
xmlns : soap*" http: / / schemas.xmlsoap.org/wsdl / soap/" 
xmlns:tns="http: / /MyServer/OntoPassport.wsdl" 
xmlns :nsl=" http: / /My Server /My WebService.xsd"> 

< types > 

< schema targetNamespace="http://MyServer/MyWebService.xsd" 
xmlns*" http: / /www. w3.org/2001/XMLSchema" 
xmlns :S0AP-ENC=" http:/ / schemas.xmlsoap.org/soap/encoding/"> 

<complexType name*" OntoPassportTo ken" > 

<all> 

<element name="name" type*"string"/> 

<element name="idCode" type="string"/> 

<element name="surname" type="string"/> 

<element name="mail" type="string"/> 

<element name="privilege" type="string"/> 

< element name="ttl" type="int"/> 

<element name="timeStamp" type="int"/> 

< / all > 

< / complexType > 

< /schema > 

< /types > 

<message name="tokenRequestRequest" > 

<part name="usr" type="xsd:string"/> 

<part name="pwd" type*"xsd:string"/> 

< /message > 

<message name="tokenRequestResponse" > 

<part name="return" type="nsl:OntoPassportToken"/> 

< /message > 

<portType name*" OntoPassportType" > 

<C operation name="tokenRequest" > 

< input name="tokenRequest Request" message*" tns:tokenRequest Request "/> 
Output name="tokenRequestResponse" message="tns:tokenRequestResponse"/> 
<C /oper at ion > 

< /port Type > 

<binding name="OntoPassportBinding" type="tns:OntoPassportPortType"> 

< soap : binding style="rpc" transport="http://schemas.xmlsoap.org/soap/http"/> 
< operation name="tokenRequest" > 

< soap: operation soapAction*"" style="rpc"/> 

<input name="tokenRequestRequest"> 

<soap:body use="encoded" namespace*" My WebService" 
encodingStyle="http:/ / schemas. xmlsoap.org/soap/encoding/"/> 

< /input > 

<output name="tokenRequestResponse" > 

<soap:body use="encoded" namespace*" My WebService" 
encodingStyle="http:/ / schemas. xmlsoap.org/soap/encoding/"/> 

< / output > 

< / oper at ion > 

< /binding > 

<service name*" My WebService" > 

<port name="OntoPassportPort" binding="tns:OntoPassportBinding"> 

< soap : address location="http://OntoServices/My WebService"/ > 

</port> 

< /service > 

< /definitions > 



Fig. 8. The OntoPassport WSDL interface 

and the resources, they do not fully exploit the semantic power and reasoning 
capabilities of emerging web applications. We have presented a semantics-aware 
approach aimed at controlling access to resources on the basis of complex asser- 
tions about subjects seeking access as well as about resources, stated by means 
of semantic web metadata standards. We have also shown how this expressive 
power can be easily accommodated by proper extensions of available XML-based 
policy languages, like XACML. While several aspects (including efficient tech- 
niques for performing enforcement) are still to be investigated, our proposal 
provides a clear problem statement and a first step toward its solution. 
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Abstract. In this article we introduce HyCo (Hypertext Composer), an 
authoring tool devoted to create semantic learning objects. This authoring tool 
uses learning technology standards or specifications to save these semantic 
objects, which will be delivered in Web e-learning environments as 
encapsulated packages in order to ensure their reusability, interoperability, 
durability and accessibility. These learning objects are closed to the Semantic 
Web field because they combine hypermedia and semantic capabilities. Our 
research work is directed to use these semantic learning objects in order to 
define learning domains for an Adaptive Learning Environment. The aim of this 
system is to provide an e-learning environment where teachers have tools to 
create didactic materials and students carry out their knowledge acquisition 
through the most suitable adaptive learning technique giving the student’s 
characteristics, the learning activities provided, and the learning objects’ 
features. 

Keywords: Hypermedia Authoring Tools; Semantic Web Applications; 
Learning Technologies Standards; IMS; E-Learning. 



1 Introduction 

The use of Internet as an instructional media not only has brought new ideas and 
thoughts around learning and teaching, but also a new conception about learning 
elements through WWW: they should be reusable, interoperable, durable and 
accessible. 

To accomplish these requirements, several education institutions had incorporated 
information and communication technologies in the learning and teaching process in 
order to increase the quality, efficiency, and dissemination of the education. 
Consequently, learning domains have been passed thru a reconfiguration process 
where defining metadata for learning objects has a central role. Metadata guarantees 
interoperation, reusability, and interchange among e-learning systems, as well as a 
cost-effective development. 
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Several LTS (Learning Technology Standards) have been defined, which are 
agreements about the characteristics a learning element should have in order to be 
compatible, interchangeable and interoperable into other learning systems [9]. 

If web-based e-learning systems fulfill a LTS, they are able to work with other 
systems (interoperability), follow-up information about learners and contents 
(manageability), generate learning objects that are usable in other contexts 
(reusability), and avoid obsolescence (durability). From students’ point of view, 
standards ensure they get the right content at the right time (accessibility) and obtain a 
variety of knowledge resources (interchange of learning objects). 

In our research work, we are interesting in defining SLOs (Semantic Learning 
Objects) that will be compliant with these LTSs and will be integrated or deployed in 
an ALE (Adaptive Learning Environment). The aim of an ALE system is to provide 
an e-learning environment where teachers 
have tools to create didactic materials and 
students carry out their knowledge 
acquisition through the most suitable 
adaptive learning technique giving the 
student’s characteristics, the learning 
activities provided, and the learning objects’ 
features [1], 

The proposed ALE architecture is 
composed of five subsystems: Learning 
Domain Model, Adaptive Model, Student 
Model, Adaptive Meta-model, and Deliver 
and Packing Model, as it is shown in Fig- 
ure 1. 

The system pretends to be an open tool that " " 

differentiates between educative contents and learning process. The ALE system 
structures its semantic elements following the IMS specifications. Namely, to 
describe Learning Objects it uses IMS Metadata [6], to describe the Deliver and 
Packing Model it uses IMS Content Packaging (IMS CP) [4], and to describe the 
learning domain model, it uses IMS Learning Design (IMS LD) [5], 

The rest of the paper is organised as follows. Section 2 gives a general introduction 
of the HyCo authoring tool. Section 3 is focused on the definition of the SLOs in 
HyCo. Finally, section 4 provides remarks and further work. 




Fip. 1 . AT,F. Architecture 



2 An Overview of the HyCo Authoring Tool 

HyCo [3] is a powerful authoring tool for educational purposes; this means that an 
author can create hypermedia educational resources with it. But also the same tool 
could be used to access to the created contents in a read-only mode by a student or 
reader. 

HyCo is a multiplatform tool. It does not force to use one concrete platform. The 
idea is that if we want the teachers use it, they should work in the context they feel 
good. The actual version of HyCo works in the wider range of operating systems, for 
this reason Java 2 Standard Edition technology was chosen as development base. 

The main goal of the HyCo is the creation of educational contents, but trying to 
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achieve an independence of the final format publication. There exits a clear separation 
between the contents and its presentation. This way the educator writes the contents 
once, and reuses them every time he/she needs. In order to achieve this goal, HyCo 
tool uses an internal XML-based format [2] to store the educational contents of the 
produced electronic books. Precisely, the HyCo XML-based format allows the 
introduction of the LTSs in this authoring tool, specifically HyCo supports IMS 
specifications [4, 5, 6] and EML (Educational Modelling Language) [8], 

The fact of separating the content and the presentation forces to offer to the authors 
a way to generate an independent result of the authoring tool. In this way HyCo has 
an output gallery that supports HTML, PDF, TXT, RTF, SVG and PS output formats. 

Following pedagogical criteria, HyCo reproduces the process that an author 
would follow to create a linear educational resource, but it channels it, at the same 
time as it organizes it, through the metaphor of the content index, adding the facilities 
for including multimedia elements as well as hyperlinks. This way a hierarchical 
structure is obtained that guides us in our creative process, which would consist in 
associating contents with each index entry, an index that may vary as the contents 
take shape, by inserting, eliminating or changing entries. In a nutshell, then, HyCo 
faithfully reproduces the process previously explained. This indexed or tree structure 
facilitates the authoring of the hypertext, but having only an index as navigation tool 
is not acceptable in order to create real pedagogical hypermedia resources where the 
student may construct its own knowledge. This way, the hyperdocuments should be 
designed in such a way as to encourage the readers to see the same text in as many 
useful contexts as possible. This means placing texts within the contexts of other 
texts, including different views of the same text [7], For this reason HyCo also allows 
associating links to the multimedia elements that compose an index entry, i.e. a 
hypertext node. Thus, the hypertext can be followed by its index-structure, but when a 
node is selected, the reader may choose navigating by an existing link. Thus, HyCo 
documents combine both content index and Web-like structures. 



3 Definition of Learning Resources in HyCo 

When organizations, schools and teachers started to use the Web as an instructional 
media, almost the only way to publish educational contents was in HTML format, 
where e-learning elements were presented without any division between content and 
its meaning. This syntactic presentation prevents to automatically extract data, not to 
mention the definition of learning elements were as heterogeneous as existing e- 
learning designs. In this context, interoperability, reusability, and interchange among 
e-learning systems are impossible. Moreover, the idea of a cost-effective e-learning 
development is far away. 

Several organizations have been working to define specifications and standards to 
design instructional elements, known as Learning Technologies Specifications. EML 
and IMS Specifications are LTSs that are now supported in HyCo authoring tool. 

As we stated before, HyCo is an authoring tool to create SLOs, which are the 
learning resources that will be available in the instructional design process that 
defines the ALE learning domain. 
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3.1 Definition of Learning Resources 

To create the learning domain model, the first step is to generate SLOs. Every SLO 
should be compliant with IMS Metadata [6], Every section of every educational 
resource or e-book created in HyCo can be converted to a SLO. 

To do it, HyCo executes a two-step process where the first step is an automatic 
process, while the second step is a manual process. In the automatic process, HyCo 
sets all the IMS Metadata elements that can be inferred from other data or that are 
liable to have default values. 

Once this process is over, HyCo executes the manual process where it presents to 
the user the elements that can not be automatically generated and/or require 
reexamination, modification, or addition. 

When the two-step process is finished a XML file is generated for each new SLO 
(each one of them corresponds to each educational resource, section, or subsection) 
and stored in an IMS Metadata SLO repository. This repository will allow us to have 
learning objects that can be attached to learning activities of the learning designs 
created in ALE. 



3.2 Definition of Learning Components 

The learning components design includes the definition of roles, learning activities 
and learning activities sequences. 

Two steps are needed to define the learning components. First the addition and 
definition of roles and learning activities, and then connect these elements by means 
of an activity sequences. 

The definition of roles includes elements as title, metadata and information. The 
definition of learning activities includes elements as title, metadata, learning 
objectives, prerequisites, description, feed back description, and so on. Also SLO can 
be included. 

In order to ensure efficiency and simplicity in the authoring process, repositories of 
learning objects and learning designs, as well a set of selectors and creators will be 
provided. For example, to define learning activities the author can choose one 
learning object (i.e. SLO) from its repository and use selectors for describe elements 
as metadata, prerequisites, and learning objectives. Moreover, these selectors will be 
used reiteratively within the different definitions, where the same elements exist. The 
most representative case of this is the metadata selector that will be used when 
defining roles, learning activities and sequences of activities. 

In addition, in the creation of learning activities, ALE will be able to propose 
learning objects taking into account the metadata stored in other elements as 
prerequisites, objectives, and so on. 

After the author has defined the learning components, the next phase is to design 
the learning method where conditions and attributes will be delimited. Finally, the 
learning components and the learning method have to be integrated into the definition 
of the learning design, where general prerequisites and objectives are added, as well 
as roles, activities (that group learning activities and sequences), and the learning 
method. 
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4 Conclusions and Further Work 

In this paper we have introduced HyCo as an authoring tool that allows the definition 
of both learning resources and learning components or SLOs, i.e. semantic 
educational resources based on XML specifications, which could be delivered in 
diverse Learning Management Systems. Specifically, HyCo supports EML and IMS. 

The success of the HyCo authoring process has been proved with three educational 
web-based systems. Two of them are drafts devoted to test the authoring tool, one 
about computer history and other one about a software engineering course. But the 
third one is a complete electronic book in hypermedia format about cardiovascular 
surgery that is formed by 14 chapters, more than 500 sections and over 1000 images. 
This book is successfully used in the lectures of this subject in the University of 
Salamanca. 

HyCo is an open system in which too much further work is going to be done, 
especially in adaptive hypermedia systems. Now the next step is to make the 
necessary changes and modifications to HyCo in order to turn it into the learning 
domain authoring tool of the ALE System. 

Also, we are working in defining two more phases for the learning domain. 
Namely, a definition of a learning style theory and the definition of the adaptive rules 
the ALE system will follow. The first will be necessary to setup the students’ 
characteristics that will be considered. The second will be helpful to define the 
adaptation rules the system will take into account to perform the adaptation of 
contents and links. 
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Abstract. While introducing the HTML standard to present information on the 
World Wide Web, the importance of being able to express the deep structure 
and meaning of the information was neglected. This has lead to some of the 
limitations of the current web (e.g. its restricted query possibilities). Work has 
started in the domain of the semantic web which tries to solve this problem by 
annotating web pages with semantic information. A crucial aspect to the suc- 
cess of the semantic web is that we have methods available to create, integrate 
and use this semantic information. In this paper, we present a new approach to 
generate semantic information by taking the annotation process to a conceptual 
level and by integrating it into an existing website design method. 



1 Introduction 

The large majority of current information available on the web is presented using the 
standard HTML format. Emphasis was put in this standard on layout possibilities but 
the importance of being able to express the meaning of the presented information was 
neglected. The lack of semantic information in current websites is addressed by the 
vision of the Semantic Web [1], This vision states that the information available on 
the WWW should be defined such that it remains usable for human interpretation, but 
also becomes usable for machines. In this way, we can solve some of the limitations 
of the current web (e.g. its restricted query possibilities, intelligent agents, ...). 

A crucial aspect to realize the vision of the Semantic Web in practice is that we 
have methods available to create, integrate and use semantic information and this, as 
much as possible, in a transparent and automatic way. As mentioned in [7], the gen- 
eration of semantic markup should be a by-product of normal computer use. A step 
towards this goal has been taken in recent years by annotation approaches such as 
SHOE [6], MindSwap [4] and CREAM [5]. While such tools solve a number of issues 
like syntactic mistakes or inconsistencies with the used ontology, a number of funda- 
mental problems still remain. The main reason for these problems is that current tools 
define a linkage between an ontology and the actual data of the website on an imple- 
mentation level resulting in a strong weaving of semantics and implementation. We 
list some of the problems we encounter in current annotation approaches: 

• Despite the introduction of supporting tools, the annotation process remains a very 
heavy and time consuming task. In addition, in most current approaches this proc- 
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ess is an additional activity and the ones that will benefit from the annotations are 
usually not the ones that should accomplish the job. Therefore, the motivation for 
performing the annotation process is low. 

• It is usually assumed that the granularity of the concepts defined in the ontology 
matches exactly the granularity of the data on the website, although this assump- 
tion cannot be taken for granted. It must therefore be possible to define a link be- 
tween semantically equivalent concepts but with a different level of granularity. 

• Most of the supporting tools only allow annotating static websites, page by page on 
an implementation level. Even approaches that support the annotation of dynamic 
generated websites (by annotating the database) create a direct link between the 
implementation structure of the database (i.e. tables and columns for a relational 
database) and concepts in the ontology. For static web pages this has as conse- 
quence that the work done for one page needs to be repeated for similar structured 
web pages and that the maintenance of the metadata becomes a heavy task with a 
huge cost. Also note that for both static and dynamic websites, every time one 
changes the implementation of the website or database, even though nothing has 
changed to the semantics of the presented data, the defined linkage between the 
web pages or database and the ontologies can be affected. 

In this paper we present initial ideas for annotating websites during their design. 
The presented approach tries to solve the problems mentioned earlier by elevating the 
annotation process to a conceptual level. It is also our belief that (whenever possible) 
the annotation is best done while designing the website, not after it is implemented. In 
this way we can take advantage of the information available during the website design 
process to ease and improve the annotation process. Therefore, we propose to inte- 
grate the annotation process into an existing website design method. Several website 
design methods have already been proposed in literature. We will use WSDM (Web 
Site Design Method) [2] in our approach as this method is well suited for our purpose 
as it proposes an explicit information-modeling step at a conceptual level. 



2 Approach Overview 

2.1 Architecture 

Figure 1 gives an overview of the global architecture of our annotation approach. The 
different phases of WSDM that are relevant for our annotation approach are at the 
left: Task Modeling, Navigational Design, Page & Presentation Design, Database De- 
sign and finally the Implementation. Our approach is integrated into the original 
phases of the WSDM design method. A short overview of each step of the WSDM 
method, together with the enhancements (if any) we made for our annotation ap- 
proach, is given below. 

• Mission Statement Specification: Specifies the subject and goal of the website and 
declares the target audience. No enhancements are needed in this step. 

• Audience Modeling: In this phase the different types of users are identified and 
classified into audience classes. For each audience class, the different requirements 
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and characterizations are formulated. Also in this step, nothing additional is 
needed. 

Task Modeling: A task model is defined for each requirement of each audience 
class. Each task defined in the task model is elaborated into elementary tasks. For 
each elementary task a data model (called ‘object chunk’) is created, which models 
the necessary information and/or functionality needed to fulfill the requirement of 
that elementary task. ORM (Object Role Modeling) is used as the representation 
language for the object chunks. For our purpose, we added an annotation process to 
the Task Modeling phase. This results in the creation of a linkage between the ob- 
ject types and roles of the different object chunks and the concepts of one or more 
ontologies. This annotation is called the conceptual annotation (arrow A in Figure 

1) because it is performed on a conceptual level. In this way we define the seman- 
tic meaning of the object types and roles used in the object chunks. This conceptual 
annotation is performed for static as well as dynamic websites. 

Navigational Design: In this phase of WSDM the navigational structure of the 
website is described by defining components, connecting object chunks to those 
components and linking components to one another. 

Page Design: During Page Design, the components of the navigational structure 
and their associated object chunks are mapped onto a Page structure defining the 
pages that will be implemented for the website. We determine which object chunks 
will be placed on a certain page. Using this step as well as the previous one (the 
navigational design) we can identify which object chunks will be placed on a page. 
This is necessary to know for the actual implementation which annotations we 
have to add to a page. 

Presentation Design: For each page defined in the Page Design a page template is 
created defining the layout of the page. This layout is defined in an implementation 
independent way. To implement the actual web pages making use of a chosen im- 
plementation language (e.g. HTML, XML, ...), an instantiation of these page tem- 
plates can be generated. For this, the templates are filled using the proper data to 
obtain the actual pages. 

Data Design: As explained in [3] we can derive an integrated conceptual schema 
from the object chunks made during Task Modeling. This integrated object schema 
is called the Business Information Model (BIM) and can be used as the basis for a 
database schema from which an underlying database can be created. The Data De- 
sign is only done when we deal with dynamically generated websites querying a 
database. For static web pages the data design step is omitted as the actual data will 
not originate from a database, but will be supplied by the designer during imple- 
mentation. For our approach, we need to keep track of two mappings: 1) the map- 
ping from the object types and relationships of the different object chunks to their 
correspondence in the integrated BIM (called object chunk mapping ) (B in Figure 

2) ; and 2) the mapping between the BIM, used as the conceptual database schema, 
and the actual implementation (called database mapping ) (C in Figure 2). In this 
way we are able to determine the mapping between the queries specified at the 
(conceptual) level of the object chunks, and the actual database. 

Implementation: In this phase of WSDM the actual implementation of a website, 
based on the models created in the previous phases, is generated. To this step we 
added the generation of the actual annotation of the website (called the page an- 
notation) (D in Figure 2). Here we have to distinguish between static websites and 
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dynamically generated websites. For static websites only the conceptual annotation 
is needed. For dynamic websites also the chunk integration and the database map- 
ping have to be taken into consideration. 




Fig. 1 . Architectural overview 



2.2 Advantages 

The goal of our approach is to add semantic knowledge to the web pages of a new to 
create website. Opposed to current approaches, which perform the annotation on the 
web page level or on the database level (for dynamic websites), we define the annota- 
tion on a conceptual level. Web designers will provide the annotation during the con- 
ceptual design. Compared to currently existing annotation methods, this approach has 
a number of advantages: 

• The annotation is implementation independent. Current methods define the anno- 
tations directly in the implementation of the website. Using our approach, an im- 
plementation will be generated (HTML, XML, ...) and changes can be generated 
without breaking the annotation, resulting in a greater level of maintainability of 
the annotation. 

• The annotation process is uniform for static and dynamic websites. In current ap- 
proaches the annotation for static and dynamic websites is done in a different way: 
respectively annotating web pages or a database. In our approach, the annotation 
step is done at the conceptual design which is independent on whether the website 
will be static or dynamic. 

• Reuse of the annotations. In current annotation methods (for static websites), if a 
certain concept is used on different pages, the annotation has to be repeated for 
each page. In our approach, the annotation has to be defined only once and the 
same concept can be reused in different object chunks. Moreover, all copies of an 
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entity used over several Object Chunks will be updated automatically if the anno- 
tation of one copy has changed. 

• Improvement of the design process. An important aspect of integrating the annota- 
tion into the design process is that it enables us to improve the consistency during 
this website design process and to speed it up by making use of the metadata al- 
ready provided. It is for example possible to make suggestions to the designer 
about information to be included based on earlier conceptual annotations made. 



3 Conclusion 

In this paper, we presented an approach for the semi-automatic annotation of static as 
well as dynamic websites. The actual annotation process is performed during the de- 
sign phase of the website. We presented the proposed approach integrated into an ex- 
isting website design method, WSDM. This design method provides us a conceptual 
model of the website that can be used to annotate (at a type level) the information that 
will be available on the website, with concepts from an ontology. This is done by an- 
notating the entities (Object Types and roles) used in the conceptual model of the 
website. Next, this “conceptual” annotation can be used to generate the actual page 
annotation by keeping track of the different transformations performed during the de- 
velopment process to derive an implementation. 
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Abstract. The Semantic Web will allow software agents to understand 
and reason about data provided by Web applications. Unfortunately, for- 
mal ontologies, needed to express data semantics, are often not readily 
available. However, common data schemas can help to create ontologies. 
We propose mappings from XML Schema to OWL as well as XML to 
RDF and show how web engineering can benefit from the gained expres- 
siveness as well as the use of inference services. 



1 Introduction 

The Semantic Web will allow software agents to understand, share and reason 
about data that is provided by Web application systems. Formal conceptual 
models, or ontologies, are necessary to express the semantics of the data. Un- 
fortunately, semantic information is not usually available in such a form, but 
scattered across documentation and various software components. Because de- 
veloping ontologies from scratch is costly and difficult, one should try to reuse 
this semantic information as much as possible. Thanks to their formal nature, 
document schemas like XML Schema provide a good basis for developing or re- 
engineering ontologies (see e.g. [1,2] for a comparison of ontology languages, web 
standards and markup languages). 

In this work, we focus on extracting semantic information out of document 
schemas and propose a mechanism to lift XML Schema to the Web Ontology 
Language (OWL). While, for example, concrete translation procedures from OIL 
or XOL to XML Schema have been developed by [3,4], we specify and imple- 
ment a mapping in the reverse direction producing an OWL ontology. In order 
to apply this semantic meta-information for reasoning on instance data, XML 
documents have to be mapped to RDF, bridging the gap between those models. 
General solutions typically require changes to XML or RDF: Melnik [5] devel- 
oped an RDF interpretation for XML documents. In [6,7] a formal model and 
an architecture have been developed that allow uniform access to both types of 
documents. We propose another such mapping that does not require changes to 
the standards and incorporates XML Schema type information. Subsequently, 
based on our two mappings, we show that the engineering of XML based web 
applications can benefit from the high expressive power OWL has to offer and 
from inference services such as classification or satisfiability checking. 
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In the following, sect. 2 proposes two mapping concepts from XML to RDF 
and from XML Schema to OWL that allow lifting data from syntax to represen- 
tation. Sect. 3 shows how reasoning techniques on such representations can be 
applied in web engineering. Finally, sect. 4 concludes. 



2 From Syntax to Representation: Mapping Concepts 

In this section, we propose a general binding of XML structured data to Semantic 
Web languages. The approach is twofold: XML documents are translated into 
RDF graphs and XML Schemas are lifted to OWL ontologies. This applies to 
all XML documents that conform to an XML Schema. 

The first part of the concept concerns the XML to RDF mapping. XML is 
a language that defines a generic syntax to store and exchange documents by 
means of a tree-based structure. Although RDF has an XML-based syntax, XML 
and RDF serve different purposes and have been developed separately within the 
W3C, which lead to different modelling foundations. 

XML is based on a tree model where only nodes are labeled and the out- 
going edges are ordered. This model originates from semi-structured data and 
databases. In contrast to this, RDF is based on a directed graph model where 
edges have labels but are unordered. It distinguishes between resources (e.g. car) 
and properties (e.g. car color) while XML does not (e.g. both would be elements). 
This model originates from knowledge representation languages such as Frames 
[8] and description logics (DL). 

To bridge the gap between both forms of data representation, we developed 
a procedure that transforms XML documents to RDF data models. In order to 
keep compatibility with existing documents and applications, this mapping does 
not require any change on either XML or RDF specifications (however, so-called 
mixed content models of XML are not fully supported). Structural differences 
of the data models represent no obstacle, as trees are a specialisation of graphs. 
We make a distinction between (1) elements that have sub-elements and/or 
attributes, and (2) attributes and elements that carry only a data type value. 
These two categories of components correspond respectively to XML Schema 
declarations associated with a complexType or a simpleType. The mapping is 
performed by the following procedure: 

Initially, an RDF resource Document is created - representing the XML doc- 
ument itself. Then, for each sub-element and attribute of the element that is 
currently processed (starting with the root element), an RDF property on the 
RDF resource created before in the previous step is created. If we encounter 
a data type component (2nd category above), its data value is represented as 
an RDF literal on the respective property. If we encounter an object component 
(1st category above), an anonymous RDF resource is created and assigned as the 
value of the respective property. Then, this component is processed recursively. 

As we also want to map XML Schema, it is desirable to transparently incor- 
porate the type information specified in the corresponding schema. To facilitate 
this, we presume that an XML Sclrema-aware processor has validated the XML 




356 M. Ferdinand, C. Zirpins, and D. Trastour 



document, which results in type information represented in a Post-Schema Val- 
idation Infoset (PSVI). We will see that each XML Schema complexType is 
mapped into an OWL class. Hence each mapped RDF resource is of rdf :type 
the OWL class corresponding to the PSVI retrieved complexType. 

The second part of the concept concerns the XML Schema to OWL mapping. 
XML Schema and OWL solve different problems: XML Schema provides means 
to express and constrain the syntax and structure of XML documents. OWL, 
in contrast, is intended for modelling the semantic relationships of a domain. 
However, there is an interesting overlap between the two, as both of them have 
an object-oriented foundation. XML Schema has the notion of class hierarchy 
and specialisation, and OWL is based on the notion of Frames. Although they 
accomplish it at two different levels of abstraction, the languages share the goal 
of defining common vocabularies and structures to support electronic exchange 
of information. Our mapping approach that complements the one seen before, 
capitalizes on these similarities. In the following, we give an overview of the 
fundamental choices. The mapping procedure of a complete XML Schema is 
composed of the mapping of its different components. 

Main Concepts: Each XML Schema complexType is mapped to an 
owl: Class. Each element and attribute declaration is mapped to an 
OWL property. More precisely, elements of simpleType and all attributes 
are mapped to an owl: Data typeProperty; elements of complexType are 
mapped to an owl : ObjectProperty. Finally, the schema root element of a 
schema is mapped to an OWL Class of name ’targetNamespace + #Schema’. 
Model Groups: Model group definitions and attribute group definitions are 
specialisations of complex types since they only contain element respectively 
attribute declarations. Hence, they are also mapped to OWL classes. 
Specialisation: In object-orientation, inheritance mechanisms represent a cen- 
tral modelling tool which is used to express ”is-a” relationships between 
classes. The literature differentiates between various types of inheritance, two 
of the most important ones are inheritance by restriction and inheritance by 
extension. XML Schema supports both of these ways by corresponding type 
derivation constructs and we both map them to rdf s : subClassOf in OWL, 
its only inheritance mechanism. XML Schema offers the substitutionGroup 
construct which specifies that an element can be replaced by a set of other 
elements in the instance document. Analog to the type derivation mecha- 
nisms, this construct can be interpreted as a way to express a specialisation 
of elements and thus is mapped to an OWL subPropertyOf . 

Type and Cardinality: In XML Schema, ’’Particles” (resp. ” AttributeUses”) 
are used to associate a type and cardinality to a local element (resp. a lo- 
cal attribute). Because these definitions have a local scope, we map them 
to the intersection of two property restrictions: one restricting the type 
with owl : allValuesFrom, the other restricting the cardinality with either 
owl :minCardinality, owl :maxCardinality, or cardinality. The two re- 
strictions apply to the same property (i.e. the one corresponding to the 
element or attribute). 
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Compositors: XML Schema offers three compositors to combine elements, 
sequence, all and choice. They are mapped to appropriate OWL boolean 
expressions. The difference between sequence and all is purely syntac- 
tic; semantically they are both conjunctions and are both mapped to an 
owl : intersectionOf constructor. The mapping of the choice compositor 
is more verbose since there is no direct equivalent in OWL to an exclusive- 
OR. Hence, it needs to be constructed with a boolean expression (with 
owl : intersectionOf , owl:unionOf and owl : complementOf ). 

Global Elements: Global element and attribute declarations are mapped sim- 
ilarly to local ones. Associated restrictions are added to the Schema class. 

Identifiers are mapped from XML Schema to URIs by concatenating the 
targetNamespace URI, the # character and the component’s local name. 
Problems can occur due to the fact that XML Schema partitions the 
targetNamespace into distinct so-called symbol spaces, one for each kind of 
definition or declaration. To prevent naming conflicts in OWL, the mapping 
process applies an appropriate renaming pattern to the affected components. 

As a detailed discussion is out of scope here, we can only briefly note that we 
also found mappings for other language constructs of less common interest. How- 
ever, because of a limited expressiveness in OWL or because the construct would 
not be appropriate, we also had to skip some non-essential language components 
like abstract, final, block, default, form, wildcards, identity-constraint def- 
initions and complexTypes derived by restriction from simpleTypes. 

3 Reasoning Support for Web Engineering 

There are a number of promising applications for the mapping concept in Web 
engineering. In particular, it allows enhancing traditional XML languages and 
tools by the capabilities of OWL reasoners. Here, we distinguish support ca- 
pabilities at design time and runtime of web applications. At design time , we 
see two principal usages for the mapping. On the one hand, ontologies can be 
extracted out of existing XML Schemas. This skeleton ontology can then be ex- 
tended using OWL expressions. On the other hand, the mapping can support 
schema design. Analogous to the use of reasoners to design ontologies [9], they 
are useful to design XML Schemas. By using owl : equivalentClass instead 
of rdf s : subClassDf for the mapping of complexType, an OWL reasoner can 
infer implicit subsumption relationships, thus identifying super- types of some 
complexTypes. This fosters reuse and limits the class proliferation when large 
number of classes are encountered. An OWL reasoner could also help to check 
the compatibility of two independently developed schemas. DL based reasoners 
are the most suited for this type of operation as they can do efficient inference 
on classes. At runtime, the XML mapping into RDF can be used to do inference 
and semantic validation of XML data. Once translated into RDF, the data can 
be classified with an OWL reasoner. The classification could lead to discover 
implicit class membership or implicit relationships between objects. Finally, se- 
mantic validation can be performed by looking for unsatisfiable concepts. 
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4 Conclusion 

We have proposed a general solution for automated binding of XML structured 
data to Semantic Web languages. General procedures have been shown to map 
XML documents to RDF graphs and XML Schemas to OWL ontologies. Subse- 
quently, supporting techniques for the engineering of web applications have been 
presented that get possible by integrating mapping results with OWL reasoners. 

To underpin the concepts, we offer a Java software toolkit that implements 
the mapping process 1 . In terms of engineering concepts, we note that we incor- 
porated the RACER DL reasoner [10] and used its inference services to realise 
a real-world e-business Web application in the context of RosettaNet [11]. 

By automatically generating formal conceptual models from semi-structured 
data, our approach supports the automated bootstrapping of ontology develop- 
ment from existing XML Schemas, speeding up the adoption of Semantic Web 
technologies. It opens up to a wide range of XML based web applications the 
expressive power of OWL as well as the potentials of inferencing services. Unlike 
most traditional techniques (e.g. hard coded validation), semantic constraints 
can be written in a formal, well-documented and reusable fashion that can be 
applied to various tasks such as semantic validation of XML instances. 
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Abstract. The recent trend in the Internet traffic is increasing in requests for 
dynamic and personalized content. To efficiently serve this trend, several server- 
side and cache-side fragment-based techniques, which exploit reuse of Web pages 
at the sub-document level, have been proposed. Most of these techniques do not 
focus on the creation of the fragmented content from existing dynamic content. 
Also, existing caching techniques do not support fragment movement across the 
document, a common behavior in dynamic content. 

This paper presents two proposals that we have suggested to solve these prob- 
lems. The first, DyCA, a dynamic content adapter, takes original dynamic Web 
content and converts it to fragment-enabled content. Thus the dynamic parts of 
the document are separated into separate fragments from the static template of 
the document. This is dependent on our proposed keyword-based fragment detec- 
tion approach that uses predefined keywords to find these fragments and to split 
them out of the core document. Our second proposal, an augmentation to the ESI 
standard, allows splitting the information of the position of each fragment in the 
template from the template data itself by using a mapping table. Using this, a frag- 
ment enabled cache can have a more fine grained level of identifying fragments 
independent of their location on the template, which enables it to take into account 
fragment behaviors such as fragment movement. 

We used the content taken from three real Web sites to achieve a detailed per- 
formance evaluation of our proposals. Our results show that our keyword-based 
approach for fragment extraction provides us with cacheable fragments that, when 
combined with our proposed mapping table augmentation, can provide significant 
advantages for fragment-based Web caching of existing dynamic content. 



1 Introduction 

Researchers have recently proposed several server-side and cache-side mechanisms to 
improve the generation and serving of dynamic Web content. Server-side techniques, 
exemplified by techniques such as delta encoding [1], data update propagation [2], 
fragment-based page generation [3,4], reduce the load on the server by allowing reuse 
of previously generated content to serve new requests. Cache-side techniques, exem- 
plified by systems such as Active Cache [5], Gemini [6], CONCA [7], and the content 
assembly technique proposed by Wills et al. [8], attempt to reduce the latency of dy- 
namic content delivery by moving some functionality to the edge of network. Similar 
trends are also visible in commercial caching and edge server products, most notably 
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IBM’s WebSphere [9] and Akamai’s Edgesuite [12]. Despite their difference in focus, 
both server-side and cache-side approaches share the same rationale, specifically that it 
is possible to view the document in terms of a quasi-static template (expressed using 
formatting languages such as XSL-FO [10] or, what is currently becoming the de facto 
standard, edge-side include (ESI) [11]), which is filled out with multiple individually 
cacheable and/or uncacheable objects 1 . This object composition assumption enables sur- 
rogates and downstream proxy caches to reuse templates and cached objects to efficiently 
serve subsequent requests and additionally reduce server load, bandwidth requirements, 
and user-perceived latencies by allowing only the modified or unavailable objects to be 
fetched. 

Although the above techniques appear promising, there are a number of issues that 
are not addressed in these current infrastructures. Even though there might techniques 
used by certain companies [12], due to their closed nature we cannot check them, and 
so to the best of our knowledge, there is no open and free method of separating objects 
from existing dynamic document, except from our existing work on DYCE [13]. Also, 
current technologies for supporting dynamic objects do not differentiate between the 
location of the objects in the document, and object itself. This makes it impossible to 
efficiently implement the situations where the object can move between different places 
in the document without changing data, which is common in certain news Web sites [17]. 

This paper describes our efforts on addressing these shortcomings. We are proposing 
two methods that should solve these shortcomings. The first is an augmentation to the 
ESI standard, the most used method for specifying the format of the templates, to allow 
the fragment locations to be specified in a mapping table that is sent with the template. 
This allows the objects to move across the document without needing to re-serve the 
template. Our second proposal, DyCA, a Dynamic Content Adapter, is a two part model 
for creating object-based content from original dynamic content. The first part extracts 
the objects from the original content, giving us the needed separation between template, 
objects, and object location by using our mapping table approach. The second part of 
DyCA delivers the content to a fragment-enabled client, like a caching proxy server. A 
Python-based fragment-enabled caching proxy, named CONCA-Lite was developed to 
allow the testing of the object extraction, and content delivery modules of DyCA. 

Our method for creating the dynamic content from the original Web content is based 
on a simple and effective keyword based object extraction technique to find dynamic 
objects inside a static Web page. The dynamic content can then be served by our DyCA 
server, which can serve fragments from the document as needed, enabling a client to 
support template and object caching. Our proposed ESI-extended format allows for 
caching of both the objects and the template and allows for object movement. This 
type of concept, where the object in the template maps, based on a mapping table, 
is, to our knowledge, introduced here for the first time. By having a fully functional 
fragment-enabled content server and client, and by testing on real world data, we have 
gotten accurate results, beyond regular experiments done in the field. These results have 
shown that our proposed method for fragment extraction based on the keywords in the 
document can allow us to cache existing non-fragmented content and achieve significant 
performance improvements by utilizing our proposed augmentation for a mapping-table 
based template. 

1 The terms objects and fragments will used interchangeably in this paper. 
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The rest of the paper is organized as follows. Section 2 describes the related back- 
ground for this field. Section 3 addresses problems with the current infrastructures. 
Section 4 shows the design and implementation of the system. Section 5 presents the 
evaluation and results of the testing of our architecture. Section 6 concludes the paper 
and discusses our planned future work. 





Document Template 



Fig. 1 . Dynamic content can be viewed in terms of a quasi-static document template and individual 
objects, which exhibit different sharing, cacheability, and freshness time characteristics. 



2 Background 

2.1 Fragment Based Caching 

One fundamental block in caching dynamic content is allowing to split up a document 
into different static and dynamic parts. By doing this, parts of the document, called 
fragments, can be treated separately rather then treating the document as a whole. Thus 
each fragment can have its own behavior allowing more fine-grained control over caching 
behavior and data sharing of the different segments of a document. Certain fragments of 
a document can be shared between different clients whereas some clients want different 
information in other fragments. Also, some parts of a document change more frequently 
than others, while other parts are completely static. By treating the document as a whole 
rather then separating it into fragments a page cannot be partially shared between users 
nor can it be partially cached. Rather, when a little part of the document changes the 
whole document needs re-fetching, and if parts of the document can’t be shared, then the 
document can’t be shared at all. For example, consider a popular customizable Web site 
with content that gets continuously updated with information such as news and weather. 
Figure 1 shows the snapshot of a personalized my . yahoo . com page, which fits in such 
an example, and what the corresponding document template and component objects 
might look like. S* and P* represent objects that are shared and private respectively, and 
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TTL captures the length of time this object remains valid. The contents of fragments on 
the page can change as new stories develop or as the weather changes, leaving other 
fragments unchanged and with no need to re-fetch the data. Users that are viewing 
sports stories can share those fragments, and if some of those users are also viewing the 
economic section, then that can be shared by all the people viewing that fragment as well. 
There currently are a number of ways to split a document into fragments and reconstruct 
it. Different approaches do reconstruction in different parts of the network, including 
the originating server, the cache proxy, and the client. Each one of these methods has a 
different way of dealing with caching and the fragments. 



2.2 Edge-Side Includes 

The ESI [11], Edge-Side Include, has currently become the de facto standard in specify- 
ing the format for templates in fragment based documents. It uses a simple XML-based 
markup format that specifies content fragments for inclusion and dynamic assembly in 
a base template. These ESI specific tags are provided as extensions to the traditional 
HTML format allowing for minimal format change to the original document format. 
It separates the document in such a way that allows for the server or proxy to manage 
the objects as separate entities. This allows for different levels of cacheability for each 
fragment, and for large amount of dynamic content to be cached. ESI also includes a 
very complete framework for conditional getting of fragments, cookie support, and error 
control. Every esi : include tag references a specific URI with a TTL, all of this is 
included in the template file which gives the layout and aesthetic look to the document. 
Thus, all the information regarding the fragments and the template are actually sent in 
the template itself. For a client to support the ESI framework all that it needs to do is 
to implement support for parsing and acting based on the template format. Thus, ESI’s 
simplicity is a very strong point. 

CONCA [7] allows for additional client information, such as the type of device of 
the client, to modify the returned content, based on a different template yet reusing the 
same objects. 



Templatel (t=t1) 











03 




02 


Ol 



template { 

<esi:include src= o3/> 
<esi:include src= o2J> 
<esi:include src= o1/> 
} 



Templatel (t=t2) 



04 



03 



02 



template { 

<esi:include src= o4/> 
<esi:include src= o3/> 
<esi:include src= o2 /> 
} 



(a) 



(b) 



Fig. 2. An example of template caching problem. 
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3 Existing Problems of ESI 

An important factor of the efficiency of fragment based documents is the method used 
to update it. One of the popular uses in fragment documents is for object movement. 
This type of behavior, as can be seen in Figure 2, is represented when one or more 
fragments from a dynamic document move between the different available positions 
on the template. A good example is a news Web site where old stories (represented as 
fragments) move down the document and new ones are added from the top. If we use 
ESI to construct our document then there are two possible ways to update the document 
so that the objects can move across the document. Since in ESI the document is split up 
into two parts, the template, and the fragments, then one possibility will be to update 
the fragments where the object moved to where it moved from. Another possibility is to 
update the template such that the updated template has the URL of the new locations in 
the template pointing to the correct fragment. 



3.1 ESI - Static Template 

One obvious solution to fragment movement, which we will call static template, is to 
update the fragments themselves, leaving the template completely static. Using our news 
Web site example from earlier, the news template points to a series of objects, and when 
a new story gets added to the page it pushes the old one out of the page. All the objects 
in the page need to expire, be invalidated, and be re-retrieved as the fragments lower 
down on the page. Considering that a document, like a news Web site, that has most of 
its objects moving around the Web page, this means invalidating most of the objects on 
the page and fetching them from the server on a regular basis. The biggest problem with 
this method is that most of these fragments are already present on the client just in a 
different location, and re-fetching them in a different place means a lot of wasted data 
transfer that could otherwise be cached. 



3.2 ESI - Static Objects 

A different approach to solve this problem, which we will call static objects, is to in- 
validate the template and re-fetch it for spatial changes, leaving the objects static for 
such a change. The fragments will still contain only dynamic data and would need to be 
re-fetched for a data change. The new template will have the object in the first position 
pointing to the new objects, and all the other positions pointing to the moved objects. 
This seems to solve the problem since the objects are unmodified do not need to be 
re-fetched. Supposedly, this saves a lot of data transfer by transferring only the new 
objects and not requiring all the objects to be re-fetched as in our earlier example. Since 
the only things that has actually changed in the template is the URL of a few of the 
fragments, this means that most of the transferred template, which would still contain 
only static data, is already on the client. Considering that the template can be very large, 
as we show later in section 5, doing this on a regular basis, and transferring all this data 
that could otherwise be cached, this is not a very efficient solution as it might seem at 
first. 
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3.3 Mapping Table 

There is no way to solve the problem of object movement in the current ESI infras- 
tructure. Either you will be invalidating many objects that really are valid, or you will 
be invalidating a template which has very little data actually modified. That is why a 
proposed solution needs to make additions to the ESI infrastructure. These additions, 
a mapping table that gets sent along with the template and an addition to the template 
format to allow inclusion of objects from the mapping table, allow for a new method 
of fragment-based caching, which we will denote as mapping table, which does not 
requires either the objects or the templates to be invalidated for spatial changes. Thus, 
when a client retrieves a template, the client will also cache the mapping table that was 
sent with the template. When this client needs to fetch a fragment from the template, 
the identifier in the template is looked up in the mapping table, and then the fragment 
is fetched from the appropriate URL. The mapping table is relatively a small amount of 
data compared to the template and object sizes. When an object needs to be moved across 
a document, from one locations in the template into another, the only thing that needs to 
be updated is the mapping table. Thus, both the template and the objects continue to be 
cached and treated efficiently, while still allowing for object movement. The mapping 
table’s small size and ease of use makes it optimal for these cases. 




Fig. 3. The general architecture of DyCA. 



4 DyCA: Dynamic Content Adapter 

4.1 Design 

As mentioned earlier, DyCA, our proposed Dynamic Content Adapter, has been designed 
to augment the existing servers to serve dynamic content efficiently. In the client-proxy- 
server model it sits on the front-end of the server and can then take existing Web content 
that is requested from the server and process it to serve the dynamic content instead. So 
when the original content changes, DyCA will regenerate its fragment-enabled content. 
This allows it to be deployed anywhere from the same location as the actual Web server 
or to the ISP level. As shown in Figure 3, DyCA is actually split up into 2 separate 
but very important parts, the Dynamic Content Generator module, and the Dynamic 
Content Delivery module. The generator module deals with taking the existing dynamic 
content and converting it into fragmented content. The delivery module can then take 
the dynamic Web pages generated by the earlier module, and serve them to the client 
appropriately.x These two modules together let us take an existing static Web site and 
easily change it into a fragment-based dynamic Web site, that can be cached properly. 
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The content generation module generates the objects and template from the original 
dynamic Web content. It uses a keyword based approach to split up the dynamic content 
into fragments. The keyword based approach works by building an XML of the Web site 
and searching this XML tree for specific tags that can signify a different object. These 
tags are set separately for different Web site based on the structure of the HTML and 
the content. By looking at popular Web sites, such as the personalized my . yahoo . com 
page shown earlier in figure 1, it is fairly simple to see the implicit fragments contained 
in the document. Fragments can be easily distinguished based on certain differences 
such as a different font or a table tag, and based on certain predefined keywords, such 
as the TV Listings headlines, or the Weather headlines. Once the tags in the XML tree 
are identified, the children XML tags and the rest of the XML content contained in the 
identified tags is extracted to create the objects. A special include tag, that has a special 
object id for each object, will be placed in the position where the object was extracted 
from the main document. The mapping table is just a list of the object ids and their 
corresponding URLs in a parse-able file. 

Additional information for each object, such as the TTL of the object, is calculated 
by looking at a long term overview of the Web site. Numerous instances of the Web page 
are collected over a regular period of time. Each instance is parsed and separated into 
the different objects, template, and mapping table by the content generation module. By 
comparing the objects and their position over the period of time, an accurate dynamic 
behavior can be seen that allows the correct generation of the mapping table to be most 
efficient with regards to object movement. It also allows the TTL for each fragment 
location in a document to be generated. Since the location in the TTL of a location in 
the template remains mostly the same, this TTL is then reused later as the TTL for each 
fragment location in the document. In our news Web site example, the sidebar listing all 
the news from yesterday will always have a TTL of 24 hours. 

The content delivery module is responsible for serving the template, objects, and 
mapping table to the fragment-enabled proxy cache. The content delivery module uses 
the data created by the content generation module. The content delivery module needs 
to implement the extensions to the protocols in order to send the data created by the 
content generation module in an appropriate way. It needs to add information regarding 
the mapping table, and so notify its client, the fragment-enabled proxy cache, when a 
request is dynamic and has a template or when it is static. By sending the mapping table 
to the client, the client can then get the static URLs of the objects to be able to access 
them from the template. The content delivery module also needs to support the client 
updating only its mapping table, so that bandwidth would not need to be wasted with 
already cached objects, or templates. This module can be backwards compatible with 
existing technologies and can support serving to both client level caching [14], and proxy 
level caching [7,12]. 



4.2 Implementation 

The dynamic content generator is program that parses existing Web sites and outputs a 
dynamic, fragment based, cacheable, Web page. Three Web sites. New York Times [15], 
India Times [16], and Slashdot.com [17], were chosen due to their dynamic nature, and 
since none of them supported any form of cacheable dynamic data. The Web sites were 
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Client Cache Server 




(a) (b) 



Fig. 4. (a) Process of initial retrieval (b) Process of cached retrieval. 



constantly monitored for changes during an extensive period covering 2 weeks. This 
data was then passed through the keyword based object extraction. Each object was 
extracted from each instance of the Web site by finding appropriate, predefined tags in 
the document. Once each object was extracted and the ESI-based template was con- 
structed, the resulting fragmented documents were compared across their time element 
to calculate the TTL and the mapping table. Object movement across the document is 
taken into account and allows for the object to not be replaced too soon, and remain in 
the mapping table. An object was considered expired once it wasn’t in any part of the 
document. Once the template, objects, and mapping table are in place from the generator 
module, the dynamic content delivery module just needs to serve them. Implemented as 
a server extension using servlets in Java and sitting on top of popular Web servers such 
as the Apache Web Server [18] or the Jigsaw Web Server [19], this module appears as 
a traditional Web server to regular clients, but provides the dynamic content ability to 
able clients. This module uses information from the generator module to build up infor- 
mation such as which pages are templates, the TTL for certain objects, and the mapping 
table. Figure 4 shows the process of events when a client request for a Web page. When 
a CONCA-enabled request from a proxy for a document is made, if the document has 
fragments then the mapping table is looked up and added as an X-CONCA-MAPPING 
HTTP header to the response. The expiration of the object is set based on the TTL 
contained with the mapping table. The body of the response is just the ESI augmented 
template. Once the cache has the template, it will go on and request the fragment as 
needed from the server. The cache can then build up the proper final document and send 
that off to the client. The servlet’s support of the If-Modif ied-Since HTTP header 
when requesting the template is crucial to the efficiency of the mapping table. When the 
template expires on the cache, the cache will request the template over again using the 
If-Modif ied-Since HTTP header. Since the template rarely changes, this will mean 
that the cache will usually get a ’304 Not Modified’ response. This response will contain 
the new mapping table, allowing the cache to update it’s mapping table without having 
to re-transfer the template. If a static document, or a static fragment is requested from 
the content delivery module, then the content delivery module will behave just like a 
regular Web server. 
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4.3 ESI Augmentation 

As explained earlier, ESI provides and extensive range of existing technology to support 
a wide range of uses in structuring a template file and specifying things like TTL and so 
forth. In trying to remain as standard compliant as possible, the ESI format was picked 
to represent the structure of the template. Yet the ESI standard only supports document 
fragments identified by static URLs, which will not suffice in our case. Thus we needed 
to augment the ESI standard by adding the esi : xconca-include tag. This tag allows 
the specification of an ID number that can be looked up by the client in the mapping 
table and retrieve the object’s static URL. 



5 Performance Evaluation 

The experimentation of our proposed method for object extraction and object delivery 
required simulating a fragment-enabled client-cache-server model. Using this model 
we can compare different types of performance for the different types of fragment- 
enabled dynamic content behaviors. The experiments we used to test the performance 
of the different caching systems targeted user-perceived latency and bandwidth usage 
specifically. 

5.1 CONCA-Lite 

The experiments in our client-cache-server model required a proxy that supported our 
proposed augmentation to the ESI and supported the dynamic assembly of the final 
content for the client. To achieve this, a simple cache proxy, called CONCA-Lite, was 
implemented using Python and its asyncore modules to create a simple extensible 
proxy. It was designed such that testing different caching methods would require little or 
no change on the proxy side. Thus, allowing us to make fair and accurate comparisons 
between the different caching methods, which are tested on the same caching framework. 
This CONCA-Lite proxy, which implemented a minimalistic version of the CONCA 
proposal [7], was then used to test the different dynamic caching methods described 
later. 

5.2 Evaluation Platform 

We simulated the client-cache-server model using three machines, all connected on a 
lOOMbit/s LAN, at most, separated by a switch. The server, a 2.0 Ghz Pentium 4 machine 
with 512 MB RAM with Linux, ran the Jigsaw Java server to host the DyCA servlets. 
The cache, a 2.4 Ghz Pentium 4 machine with 1 GB RAM with Linux, ran the CONCA- 
Lite. The client, a 2.2 Ghz machine with 5 12B of RAM with Linux, ran a Python-based 
simulation of a client accessing Web pages in a predefined order. 

We modeled 4 different caching behaviors using our experimentation. To test each 
approach, the client was set up to request the Web page of the server from the cache at 
request intervals of 10 seconds, for a total of 10 minutes. The cache would check to see 
if it has the needed document, request the document from the server if it needs to, and 
return the document to the client. When testing a caching behavior that has a fragment 
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enabled template, it would request all of the objects in the template, it would construct 
the final document, and return it to the client. Thus the client does not need to implement 
anything beyond the standardized HTTP protocol. The first method, using no fragment 
caching, was implemented by disabling caching in the proxy, and having the server send 
the original Web page. This is consistent with the behavior of real dynamic content using 
static pages in today’s Internet, due to cookies, and other such information, that render 
a page uncacheable. In the second method the template of the document remains static, 
while the fragments of the objects are updated for content change. The template was 
cacheable for the whole testing session, while the objects were cacheable for as long 
as their TTL was valid. In the third caching behavior, the template is updated when a 
fragment moves between locations in the document, and the objects change due to data 
changes only, and not spatial changes. The last model represents our proposed mapping 
table approach. When the template is returned a mapping table is returned with it in 
the HTTP header, the proxy can then cache the mapping table, and update the mapping 
table when a spatial change happens. The template remained static for the testing session, 
while the objects only changed for data changes. 




Fig. 5. Evaluation results: (a) total data transfer between server and cache, and user perceived 
latency for: (b) New York Times, (c) Slashdot, and (d) India Times. 



5.3 Experimentation 

Three Web sites. New York Times [15], India Times [16], and Slashdot.com [17], were 
used for testing each approach. Two types of measurements were taken during the testing 
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to evaluate the performance, the total amount of data transfered between the client and 
the server, and the user perceived latency per user request. The first type of measurement 
is important to show what method performs best as a cache, with the least data transfer 
between the cache and the server. There is no need to check for the data transfer between 
the client and the cache, since in all four models it should remain roughly the same. 
The generation of the final non-fragmented Web page by the cache that gets sent to the 
client, and the method it is updated, is what changes between each method. The second 
measurement type is important to show how an improved Web caching architecture will 
benefit the client as well, and not only the server. 



5.4 Results and Analysis 

Figure 5(a) demonstrates the total bytes used in transferring data in all of the methods 
mentioned. As can be seen from the graphs, using a static template and updating the 
objects to support object movements requires considerable more data transfer between 
the server and the proxy. This is the method that is most commonly used today in dynamic 
Web sites. Using static objects and a dynamic template to achieve a dynamic Web page 
might seem efficient enough when looking at the bytes transfered, but, as we will see later, 
this efficiency is lost when looking at the user perceived latency. There is no comparison, 
though, between any of the dynamic methods and using traditional static objects. The 
amount of data transfer when not using a fragment based architecture can be more then 
10 times the amount of data transfer when using a good fragment based caching. When 
using a mapping table to transfer the data, the amount of bytes transfered is considerably 
smaller. In fact, the total bytes transfered with a mapping table is little over the total size 
of the initial site and the size of the changes, meaning a minimal amount of wasted data 
is being transfered. This should considerably reduce the server load when using such 
an architecture. Figures 5 (b)(c)(d), show the user perceived latency in seconds for New 
York Times, Slashdot, and India Times respectively. This figure shows that the user has 
to wait the least amount of time for the Web site when the mapping table architecture 
is used. With regard to user perceived latency, the static object method’s performance is 
almost as bad as the performance of using regular static Web pages. The only method that 
comes close to the method of using a mapping table is the static template method, which 
as we saw before performed badly when looking at the amount of bytes transfered. This 
type of optimization is very important for the client so it may receive its data in a timely 
manner, especially clients that use slower connections such as dialup. Otherwise, from 
the users prospective, the actual retrieval of the page is slower. From these figures we 
can conclude that the mapping table method has performed better then all of the other 
proposed methods in all of our tested fields. 

We can see some unaccounted behavior in how the 2nd and 3rd method flip in their 
efficiency between the results of the amount of bandwidth used, and the user latency 
done. When the template is static and there is no mapping table, the objects get transfered 
at a higher request rate since the proxy can’t cache them due to object movement. This 
extra data transfer has little effect on the user perceived latency in our testing environment 
due to it being a high speed network. Yet this extra data transfer is significant in terms 
of amount of data transfered, as can be seen in the earlier figure. Since the only thing 
that needs to get updated every once in a while is the transferring of the template, in 
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terms of latency, this is very close to transferring a mapping table at about the same 
interval. Yet by looking at Table 1 you can see that the template is considerably larger 
then the mapping table in most cases. This is what causes the large amount of bytes to be 
transfered. Had we artificially slowed down the network, the user perceived latency for 
static objects would have been much greater. In the case of static objects, the templates 
is considered dynamic, and gets updated every time there an object moves. Every such 
template fetch requires the cache to re-parse the template, which can be very large as 
seen in Table 1, and recheck it’s cache for every single object, in some cases this requires 
the cache to send HTTP request to see if the data was modified. This type of overhead 
causes the extra latency seen in the graph. 



Table 1 . The comparison of template and object sizes for the different Web sites. 





New York Times 


India Times 


Slashdot 


Template Size 


17 KB 


15 KB 


1.7 KB 


Avg. Object Size 


3.6 KB 


4.8 KB 


0.6 KB 


Mapping Table Size 


1.0 KB 


0.8 KB 


2.2 KB 



6 Related Work 

Dynamic Web content delivery have increasingly becoming an important Web engineer- 
ing issue as more and more Web content are generated in a dynamic and personalized 
way. Fragment-based techniques have received considerable attention from the research 
community in recent years [2,3,4,8,14]. Most of these approaches either assume the 
fragment-based content is served by Web server automatically, or look at server-side 
caching only. 

To our knowledge, few of existing work discuss the manner of how to generate 
fragment from existing legacy Web servers without server-side information. One of 
the first effort is DYCE [13], which is model-based dynamic Web content emulator. 
Recently, Ramaswamy et al. proposed a novel scheme to automatically detect and flag 
fragments [20], which share the similar goal of this paper. However, there are three 
differences between us: First, although both of our work intends to automatic detection 
of fragments, our keyword-based is simple and easy to implement, while their approach 
is complex and has theoretical analysis. However, which one is better is still not clear. 
Second, in our work we focus on engineering implementation of DYCA, while their 
work focuses on automatic detection. In this sense, their work is a good complement 
to DYCA. Third, the mapping table based fragment delivery proposed in this paper is 
novel. 

Edge Side Includes [11] is becoming one of the foundation blocks in specifying a 
common format and method for fragments and templates in this field. It is popular among 
many different existing methods. Naaman et al. [21] have done studies comparing ESI 
to delta encoding, finding ESI to have possible performance advantages. 

Automatic detection of templates from Web pages has been studied from data mining 
field as well [22,23]. They discuss the problem of template detection through discovery 
of pagelets in the Web pages. However, our work differs from the work on template 
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detection both in context and content. First, the work on template detection is aimed 
towards improving the precision of search algorithms. While our aim is improving dy- 
namic content delivery. Second, only template is interested in their work, while we care 
both template and fragments. Therefore, the method used in these two approaches are 
different too. The method presented there to finding fragments is done based on amount 
of hyperlinks present in certain parts of the document. They do not build up an XML 
tree, nor treat anything more then hyperlinks, unlike we have done in our approach. This 
method applies better to search algorithms rather then to dynamic fragment extraction. 

Our current research differentiates from earlier work done on DYCE [13], the Dy- 
namic Web Content Emulator. With DYCE, we were attempting to build up general 
models for usage to describe the behavior of current fragment-based caching. Although 
it looked promising at the time, it generated too many objects and didn’t match up to 
actual real world designs. Our current research was an attempt to try and continue that 
same research using real world Web sites so as to get more correct results. 

Other research groups [24,25] have also defined other criteria for finding objects in 
documents. While they have focused on content of the fragments and of the Web pages 
themselves, we have focused on their existence on a spatial, and location axis in the 
document. 

7 Conclusions and Future Work 

We have shown our proposed solution for keyword based object extraction, and object 
delivery. We have also explained our proposal of augmenting the ESI to include support 
for a mapping table. We have implemented these proposals into DyCA and then by taking 
actual Web pages and running through DyCA’s keyword-based extraction to transform 
them into fragment-enabled content we have been able to run simulations between our 
sample proxy and the DyCA adapter. These simulations allowed us to compares our 
proposals to the current available methods of serving dynamic content on the Web. Our 
keyword-based approach allows for creation of dynamic content in such a way as to 
maximize the cache-ability of the content in a fragment-enabled caching system. Using 
the mapping table approach in the cache proxy which, according to our results, will 
give the best performance for both the server and the client, together with our DyCA 
adapter we can effectively cache in an efficient way Web sites that currently use non- 
fragmented content. Currently our DyCA and CONCA-Lite implementations are very 
young, and could still be further optimized. Our future work consists of continuing testing 
of these implementations to further refine the design of our CONCA prototype [7], which 
incorporates a novel design for efficient caching of dynamic and personalized content. 
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Abstract. In this paper we would like to present and describe SIE, a transpar- 
ent, intelligent Web proxy framework. Its aim is to provide efficient and robust 
platform for implementing various ideas in broad area of Web Mining. It enables 
the programmer to easily and quickly write modules that improve pages on that 
site according to personal characteristics of the particular user. SIE provides many 
features including user identification, logging of users" sessions, handling all nec- 
essary protocols, etc. SIE is implemented in OCaml - a functional programming 
language - and has been released on GPL. 



1 Introduction 

We live in the era of information. The rapid development of computer and communication 
technologies enabled people to quickly exchange data at a low cost. Probably the most 
popular source of information is the Internet, and the most commonly used service is 
WWW. HTML pages are a universal way of publicizing knowledge, but it is very difficult 
to find the exact piece of information we are looking for. In our paper we would like to 
focus on one given Web site, containing various pages. The webmaster always tries to 
optimize the structure of the service in order to help users navigate. But different users 
have different preferences. When reading a page one user may next want to see page X, 
another one page Y, etc. When typing a keyword into the search engine, e.g. “chaos”, one 
user wants to read about mythology, another one about fractals, and so on. Sometimes 
a user does not know which page exactly she is looking for, because she had not visited 
it yet. One static structure will not fully satisfy all users’ needs. 

That is why dynamic page personalization is so important. It is often possible to 
predict the interests of the user, analyzing for example the history of her choices. In this 
case it would probably be helpful for the user if she was provided, on the page she is 
currently using, with the most important links. It may be even more useful if she had 
some links to pages similar to the current page to minimize time spent on searching. Of 
course the better the algorithms to predict interests and to find similar pages the better 
the page personalization process. 

The main problem with algorithms which try to understand human behavior and 
predict users’ actions is that it is extremely difficult to develop them only theoretically. 
Humans are often irrational and their language is ambiguous. On many pages consider- 
able amount of information is not only in written words but in pictures, animations or 
even sounds and these are still nearly impossible to analyze automatically. That is why 
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most algorithms are heuristics based on empirical experiments and developed through 
testing. To advance the level of research in this field, scientists need to have possibility 
to test their algorithms and to tune their parameters quickly and cheaply. 

Having made this observation we have decided to create a simple, yet powerful 
framework for constructing intelligent Web proxies. We called it SIE - Site Improving 
Engine and we wanted it not to be limited to personalizing alone - hence “Improving” 
and not “Personalizing”. SIE relieves module writers from re-implementing user iden- 
tification, network protocols handling, collection of statistics, etc. SIE gives users the 
opportunity to focus on the concept alone and enables them to quickly and comfortably 
write a new testing module. 

In addition we include a few interesting modules showing the variety of SIE features. 

It is worth mentioning that SIE and the modules are written exclusively in OCaml 
functional language. It gives the programmer many interesting possibilities characteristic 
for functional languages and provides very good performance. SIE has been released on 
GPL. Sources can be found at http : //sie .mimuw . edu.pl. 



2 Solution 

We would like to present a framework supporting programmers in creating, changing, 
testing and fine-tuning intelligent Web proxies - including Adaptive Web systems. Our 
system - SIE - implements basic features, essential for intelligent Web proxy to function 
properly. In this section we will mention all these features and in the next one we will 
present related work. Further we will discuss the architecture of the system. Then we 
will present already implemented modules, their functionality, some theoretical analysis 
and the way they are integrated with the SIE framework. In the last two sections we will 
discuss possibilities of further development and summarize the paper. 

SIE is implemented as a proxy server, transparent to the end user. It intercepts all 
HTTP messages that are sent to the server (requests) and its responses to the user. 
Our system interprets these messages and identifies the user. The identification process 
consists of two stages: when the user first sends HTTP request, the system finds out 
if she already has special cookie set, containing a unique ID value. If she does, SIE 
can identify and associate current session with the stored history. If she does not the 
system provides her with a new ID number. In the second stage, when the user fetches 
WWW pages, SIE does not employ the cookie mechanism. Instead it rewrites all the 
links pointing to the server, which are on the pages sent to the user, in such a way 
that they uniquely identify the user and the link followed. Uniqueness is achieved by 
generating 256-bit numbers and storing generated values in hash tables, which are then 
periodically purged of stale entries. For example the link pointing to http://www.i 
cwe2004.org could be rewritten as: http : // sie .mimuw. edu.pl?SIE_SESSION=AF 
387GZ2&SIE_LINK=2YUZ19A0&SIE_0RIGINAL=www. icwe2004.org. The purpose of 
each parameter is summarized in Table 1. With this method we can identify the user 
throughout each session, even if her web browser does not support (or has disabled) the 
cookie mechanism. 

The core of the whole system is constituted by modules. A module usually consists 
of two parts: offline and online. Online parts of every module is invoked whenever 
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Table 1 . Parameters added by SIE 



Parameter 


Purpose 


SIE_SESSION 


Identifies the session (and thus the user). 


SIE_LINK 


Identifies the exact link which has been followed 
by the user - the page which contained the link 
and the page the link was leading to. 


SIE.ORIGINAL 


Original link - in case the link is followed after 
appropriate hash-table is purged (e.g. if it was 
stored in bookmarks) 



a request or response is processed by SIE. Upon registration in the framework the online 
part specifies: 

- supported content types (e.g. text/html or image/png) (in case of text/html 
the module may also request a parsed HTML tree instead of pure text) 

- callbacks provided (request modifier, response modifier) 

- priority (used to determine the order in which modules are called) 

When the HTTP message contains a HTML document and there is at least one 
module which requested to receive parsed HTML documents instead of pure text SIE 
interprets HTML and constructs a parse tree, which is a very convenient form for further 
analysis and modifications. 

When modules are called back by SIE , they are provided with all the data they need 
in an appropriate form: 

- HTTP parameters 

- parsed HTML contents (when appropriate) 

- user identification 

- current trail in the traversal tree of current user 1 




Fig. 1 . Example traversal tree representing user session. Solid lines are regular clicks. Dotted lines 
represent pushing “Back” button in the browser 



Modules can modify HTML tree using received information - e.g. choose an ap- 
propriate model of user behavior basing on the user’s ID, find out which pages may 

1 Formally this would be defined as the shortest path from the root to the current node in the tree 
(e.g. in Figure 2 when user visited “C” current trail would be given as list: A — » B — > C) 
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potentially be useful for the user and insert appropriate links into HTML tree. At the end 
SIE deparses HTML back to plain text and passes this new version of the document to 
the original destination. 

Other content may be modified as well - e.g. images may be scaled down to size 
suitable for small, portable displays found in PDAs 2 or mobile phones. Modules may 
even generate responses themselves - allowing for custom pages generation. 

Before processing the request all important properties of the HTTP request are 
logged. Apart from the fields found in CLF 3 , there are two fields which together make 
up the strength of SIE. These are: 

- user ID 

- previous node in the traversal tree 

Both of these fields are taken from the rewritten link (described above), which allows 
any module written for SIE to easily obtain traversal tree similar to the one presented in 
Figure 2 from logs. Constructing trees from logs is usually done by module’s periodically 
executed part 4 , which can prepare aggregated data for the online part. A traversal tree is 
a natural representation of behavior of a user visiting a Web site. Many papers dedicate 
whole chapters to techniques of extracting information about “user sessions” 5 and 
“episodes” 6 from a CLF-compliant log (e.g. Apache access log). Users of SIE are 
relieved from reimplementing those algorithms and can focus on their modules alone. 

We are fully aware that sometimes CLF logs are the only source of information 
available but we also feel that widespread use of systems like SIE can quickly change 
this situation. Deploying SIE in front of a Web site - without any additional modules - 
can gather information useful for analyzing the site and testing new algorithms on it. The 
next step would be implementing a module which would use data collected previously. 

SIE can be easily scaled thanks to its cluster architecture. It is able to distribute 
the servicing of different clients to separate computers in the cluster. It enables the 
administrator to increase performance with the growth of a Web site. In addition, thanks to 
the Watchdog function, the system is safe and easy to maintain. It automatically disables 
the parts which do not function correctly, or shuts itself down completely when the 
whole cluster has broken down. In such cases the WWW service functions unhindered, 
but without any additional features provided by SIE modules. 

SIE has been tested on the server of the Programming Olympiad (http : //sio .m 
imuw.edu.pl). We have collected some statistical data concerning the processing of 
requests by SIE. On average the preparation of the request (HTTP parsing, creating 
threads, etc.) took about 1% of the total processing time of the request. ESEE module 
took 2%. Then the request has been sent to the web server and 88% of processing time 
was spent on waiting for reply. Afterward BM used 5%, and the remaining 4% was used 
by SIE to prepare the reply for BM, and to send it back to the user. These results prove 
that SIE has a small time overhead over the web server and LAN connection. 

2 Portable Digital Assistants 

3 Common Log Format, de facto logging standard in WWW servers 

4 We call these parts offline analyzers 

5 Defined in [17] as “a delimited set of user clicks across one or more Web servers” 

6 Defined in [17] as “a subset of related user clicks that occur within a user session” 
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During the tests (about 3 months) the system was stable and fully functional all the 
time. 

Unfortunately, due to organizational and financial constraints no tests of the “intel- 
ligence” of the modules have been performed. Such tests should be done on a large Web 
site, and should include some kind of opinion poll in order to estimate the effectiveness 
of our ideas. We hope we will manage to realize large-scale tests shortly. 



3 Similar Systems 

An impressive review of implementations of Web Usage Mining systems has been given 
by Robert Cooley in his PhD thesis [9]. It includes a systematic classification of reviewed 
systems into five categories: 

1. Personalization 

2. System Improvement 

3. Site Modification 

4. Business Intelligence 

5. Usage Characterization 

SIE with its current suite of modules ( Adapter and SEE), would probably fit into “Per- 
sonalization” and “Site Modification”. Adding other modules (as described in section 6) 
could spread SIE also to other categories. In [7] authors present system called WebCAN- 
VAS which analyzes Web server’s logs and displays visualization of navigation patterns 
on a Web site. It is accomplished by automatic clustering of users and manual clustering 
of pages on the Web site into categories. For every cluster of users, navigational patterns 
between categories are shown. These patterns represent habits of Web site’s users and 
can be used for improving high-level structure of the site. 

In [13] the author has presented IndexFinder - a tool which assists webmasters in 
adding so-called “index pages” to the site. An index page consists of links to similar 
pages. IndexFinder employs conceptual cluster mining to cluster pages not only visited 
together, but also having similar content. Proposed index pages are presented to the 
Webmaster which chooses if they should be added to the site. 

Corin Anderson in his PhD thesis ([4]) described two systems: Proteus and Mon- 
tage. The former is used to adapt Web pages to the needs of electronic devices with small 
displays, such as PDAs or modern mobile phones. Montage, on the other hand, does 
not modify content. Instead, it builds personalized web portals, consisting of content 
and links from sites the user had visited previously. 

IBM has created its own framework for creating Web proxies called WBI - Web 
Browser Intermediaries 7 . Currently WBI is part of the WebSphere Transcoding Publisher 
and the Development Kit is no longer available for download. [5] introduces concept of 
intermediaries as “computational elements that lie along the path of a web transaction”. 
This paper also describes WBI as a framework for building and running intermediaries. 
WBI supports five type of intermediaries: request editors, generators, document editors, 
monitors and autonomous. WBI, being a framework for creating intelligent proxies is, 

7 previously Web Browser Intelligence 
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in many aspects, similar to SIE . The main difference between them is the placement of 
the system between user and the Web server. As described in [6] WB1 is placed between 
the user and the Internet (all servers), whereas SIE has been thought as a proxy between 
the Web site and the Internet (meaning here users visiting the site). Additionally, SIE 
includes several features (briefly mentioned in Section 2 and described in more details 
in following sections) which would have to be implemented as modules in WBI. 



HTTP request 
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(or on separate 

machine) 
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Web server 
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Fig. 2. SIE architecture. 



4 Architecture 

As mentioned before, SIE is divided into two parts with different functionality: 

Online - this part serves the client directly, analyzes the information flow between client 
and server, updates the log file and invokes online parts of modules. 

Offline - this part is active only periodically - it performs some time-consuming anal- 
yses for the online part. 

SIE is just a framework to run special modules, which constitute the core of an intel- 
ligent Web proxy. Usually every module consists of an online part s , which is executed 
for every request, and an offline part - the analyzer. 



4.1 The Online Part 

To make SIE as robust as possible, it is crucial to have the lowest possible overhead in 
request processing and achieve maximum throughput. That is why it is very important 
to make as many calculations as possible in the offline part, and to use the computed 
results online. 

When a request is received by SIE it is processed by Request Broker , which chooses 
a Box and forwards the request to it. There may be many concurrently running Boxes, 



Sometimes we will use the term module - when we do, it will be clear from the context what 
we are referring to. 





SIE - Intelligent Web Proxy Framework 



379 




Fig. 3. Processes in online part. Arrows indicate information flow (data, document, message, etc.) 



each of them on a different computer in the network. We have implemented a cluster 
architecture, which means the workload is divided among several computers functioning 
in parallel, all of them performing the same task. Performancewise, it is crucial to send all 
requests from a particular user to the same Box during one session. Otherwise computers 
in the cluster would have to utilize some kind of a shared memory which would hold all 
information about sessions. This could easily became a bottleneck of the whole system. 
Therefore we have decided that Boxes should run completely separate, and logs they 
generate should then be combined into one big log periodically by Gatherer (e.g. via 
NFS). 

Gatherer is an external program, which reads logs generated by different Boxes and 
outputs a combined log. All concurrency issues emerging from accessing one shared log 
file in Boxes (lock contention, network issues originating from the use of a distributed 
file system, etc.) are thus avoided. 

On the other hand, there has to be centralized configuration so that every module that 
runs inside Box uses the same model and parameters. This task is fulfilled by Overseer 
- a simple in-memory database. Analyzers insert models, needed by online parts, into 
the Overseer. Some modules may query Overseer about specific information they need 
on demand and some read the current version of the model contained in Overseer when 
Boxes are starting up. Later, when new models are calculated by offline analyzers, 
a special signal is generated, which informs all Boxes that objects in Overseer have been 
changed. Upon receival of that signal modules can update local copy of the model. As 
a result all Boxes have up-to-date versions of model. 

At the beginning of the processing of every message, the Box logs appropriate prop- 
erties of request (as described in Section 2). Then the request is passed to registered 
modules, which may modify it or even generate response without involving the WWW 
server. SEE 9 (a personalized search engine and SIE control center) takes advantage of 
this functionality. SEE checks whether the request refers to Web site’s search engine, 

9 Details concerning the example modules can be found in the next section. 
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When it does - a response is produced and returned to user. Otherwise request is for- 
warded to the WWW server. Similarly, the check if the request is for the SIE control page 
is conducted. Control page is a place where the user can modify individual parameters 
for different modules. For example, she can set how many links will be added by BM to 
every page or set the favored search criterion described in section 5.2. 

When response arrives from the WWW server, the same Box which processed the 
request modifies all the links found in the HTML document sent in the response, so 
they uniquely identify user and her session. Additionally a parse tree of the HTML 
page is constructed if any of the modules uses this form. The tree is then passed to all 
active modules. In current implementation, only module called BetterMaker 10 uses this 
functionality and adds personalized links to every page. BetterMaker uses data prepared 
by eXPerimenter, an offline analyzer described below. 

4.2 The Offline Part 

Offline parts are run periodically (e.g. using standard UNIX cron daemon) during low 
system load or on a computer dedicated to this task. The main idea is to perform some 
calculations using a log produced by Gatherer, which would be impossible or too ex- 
pensive to do online. In addition SIE provides the modules with the current, analyzed 
content of the web server. A special program called Robot is responsible for preparing 
this data. 




Fig. 4. Offline part as a support for online part. Arrows indicate information flow. 



Offline parts should analyze the log and information about all the pages on the site 
in order to produce data, which would be useful for their respective online parts. For 
example, running an analyzer which would generate a model of Web site users’ behavior 
every night would make the site truly adaptive, as the model would be updated daily. 
Currently there are two analyzers implemented: 

10 See next section for more information. 
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eXPerimenter - analyzes traversal trees generated by Web site users. Then it generates 
a model for BetterMaker (the online counterpart), which uses this model online to 
personalize pages (by adding potentially interesting links to them). 

A-SEE - analyzes features of pages which have been visited by all users. E-SEE (the 
online counterpart) uses generated model to personalize search results for users by 
appropriately reordering links returned by search engine. 

A more detailed description of the example modules can be found in the next section. 



5 Modules 

5.1 Adapter 

Adapter consists of two parts 

- eXPerimenter (XP) - offline analyzer 

- Better Maker (BM) - online module 

The most visible to the end user is a little table with links predicted to be most useful to 
her. BM as an online module modifies responses sent back to the user and adds the table 
to the page. To generate this table, BM uses a model prepared periodically (e.g. daily) 
by XP. Model is stored as an object in Overseer and therefore when a new version is 
generated by XP it is automatically updated in all computers in the cluster. Number of 
links generated by system can be controlled by the user through a special control panel. 
In current implementation, the number of links chosen by the user is not stored in any 
persistent storage, so it is lost upon restart of SIE. 

To effectively render such table two problems had to be solved. First, basing on 
previous traversal patterns of users of the Web site, given current user’s trail, the algorithm 
has to predict which pages the user might visit next and with what probability. We 
chose for that task error-pruned Selective Markov Models described in [10]. The model 
contains k distinct Markov models, where k is the maximum episode length taken into 
account, fc-th Markov model contains probabilities of visiting page j, having previously 
visited pages ii, ■ ■ ■ , ik- For k = 0 the model reduces to unconditional probability 11 
of visiting given page in the Web site. With the growth of k, the model would grow 
enormously, so priming technique had to be applied. Currently, it is done using a subset 
of logs, which is not used for calculating Markov models. For details on the exact method 
of overall error priming please refer to [10]. 

Attributing web pages with probabilities is not enough. Some pages may be buried 
down in the site’s structure (i.e. to reach them user has to click on many links) whereas 
some others may be easily accessible. To compensate this, BM ranks links using expected 
number of saved clicks, i.e. the product of probability of visiting the page the link points 
to and number of clicks that would be saved had the link been put on the current page. 
To estimate this value, BM uses MinPath - a simple recursive algorithm given in [2]. 
The algorithm takes as an input current user’s trail and the model generated by XP and 
returns list of links ranked by the expected amount of saved clicks to the user. Carefully 

11 To be exact instead of probability we use frequency - the maximum likelihood estimator. 
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selected maximal recursion depth and great OCaml run-time performance allows for 
executing MinPath for every response which is sent to the user. 

Currently, Adapter does not distinguish between users - i.e. the same model is used 
for every user visiting Web site. Of course this approach may cause poor personalization 
on large sites with many different types of users. In such situation, a more sophisticated 
model needs to be used. We discuss possible improvements in the next section. 

5.2 SEE 

Another module is SEE - a search engine which aims at personalizing search results. 
More precisely, even though all searching users receive the same list of links, they get 
them in a different order. The order is set by SEE’s knowledge about a specific user. 
To illustrate this, let us refer back to the example mentioned in the introduction. Let us 
assume that the user is concerned about “chaos” meaning a mythological phenomenon. 
Therefore she should find pages on ancient gods before those concerning fractals. First 
the SEE analyzer (A-SEE) indexes the Web site’s resources rating each page according 
to a number of criteria (e.g. amount of text and pictures, number of links, etc.). The value 
of each criterion is represented by an integer between zero and ten. The criteria vector 
which is thereby computed describes the characteristics of each page. 

SIE is then employed to provide the history of the user’s searches. Not only does 
SEE focus on the keywords the user is searching for, but, more importantly, it takes 
into account which pages she chooses from the results suggested. This analysis shows 
which criteria are important for this particular user when she is looking for this particular 
keyword. It works like this: when the user looks for a word some results are provided 
by SEE or by any other search engine. The user clicks on one of the links provided. 
SEE assumes the user has chosen this particular page because she prefers it for some 
reason. After a period of time, the analyzer computes an arithmetical mean for the criteria 
values of such chosen pages. This results in SEE obtaining a set of weights for each user- 
keyword combination. These weights indicate which criteria are important (and to what 
extent) to this particular user when she is searching for this particular keyword. 

The analyzer’s task ends here. SEE comes back into action whenever the user searches 
the Web again. The resulting list of pages is sorted according to the criteria earlier 
identified as the ones preferred by the SIE user. More precisely, each resource containing 
the keyword has its criteria vector. For each page SEE multiplies this vector by adequate 
(for this particular user and keyword) weights and, thus, the ranking is computed. 

The more the user searches, the wider SEE’s knowledge about her and, thus, the 
more accurate the search results the user receives. 

To provide the user with more control over her searches, SEE allows her to choose 
one criterion to be used individually. Should the user employ this feature, her lists of 
links will always begin with pages favored by the criterion. 



6 Possible Improvements 

SIE is in an early development stage and there are many features still to be added. 
For us, it is most important to develop SIE itself as a platform for building intelligent 
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Web proxies. However, we have also a few ideas for the improvement of the already 
implemented modules and adding of new, equally interesting ones. 



6.1 SIE Itself 

SIE is a framework created to aid the programmers. This is why it is crucial to develop 
additional technical documentation, tutorials, easy and well-commented example mod- 
ules, etc. to make learning SIE as easy as possible. In the future, we are planning to 
create a graphical system to automatize basic tasks or to enable them to be performed 
by mouse drag-and-drop operations. 

On the other hand every computer system should be easy to install and maintain. That 
is why we would like to add an automatic installer as well as create ready-to-use compiled 
packages for MS Windows and popular Linux distributions. In addition, a graphical user 
interface is needed for administrative purposes. It would be also very useful to enable 
the administrator to load/unload the modules without restarting the whole system. To 
accomplish this the usage of Overseer has to be enhanced. It can be used to provide 
communication between central administration console and Boxes. The infrastructure 
is present and working (i.e. the Overseer itself) but there is no code in Box that would 
allow for remote administration and feedback (e.g. sending of warnings and system logs 
describing error conditions). 

In order to make SIE used in practice, we must improve the graphical aspect of our 
system. Elements added by our modules are readable, but they are behind the aesthetic 
standards imposed by modern HTML documents. 

6.2 Modules 

New modules. We hope to extend SIE by writing new modules ourselves and to encour- 
age others to contribute their ideas as new modules as well. Currently, we see immediate 
need to add two modules which would show: 

1 . Top k most popular pages 

2. k most recently added pages 

We are also developing a module to record and save user session (as in a sequence 
of user clicks) as a program in WTL. WTL is a new script language, developed by us 
specially for describing user behavior on a web page. Such a program can be executed 
later, simulating user actions. This simulation could be used as a test, resembling real 
scenarios of Web site usage allowing to measure Web server’s performance or find broken 
links. It can be also used to automatize some routine tasks done using a HTML interface. 



Adapter. As mentioned in section 5. 1 is a fairly simple module, which was implemented 
rather as proof of a theoretical concept than a module intended to be used in reality. Many 
features can be, however, improved or added. 

First of all, BM constructs - and XP uses - only one model. For large Web sites it 
is obvious that no single model could be appropriate for all users. Therefore, basing on 
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clustering of users. Adapter has to use many models, one for every user cluster. Possible 
approaches to user clustering are described e.g. in [7] and [ 12], 

Another technique, which could prove useful for Adapter, is page clustering. Basing 
on words (terms) contained in the documents from the Web site, the module could 
group those documents into clusters of pages with similar content. Alternatively such 
classification could be done manually or semi-automatically (with the help of someone, 
who would provide keywords for every page). Especially appealing in this context seems 
to be the algorithm called Concept Indexing (described in [ 1 1 ] ). For every page, it devises 
a list of terms (called concepts ), which best describe the page’s content. Having concepts 
attributed to every page, it is possible to create, for each user, a list of concepts she (or 
cluster of users) is interested in. Such information can be valuable from the marketing 
point of view (directing advertisements or communication to the user) and can also help 
resolve the problem of new pages - when a new page is added to the Web site it is not 
added to as a suggested link by BM because it is not yet seen in logs. With the help 
of concepts, BM can find all users potentially interested in reading the new page, and 
include link to the page on pages viewed by them. 

Additionally, concepts could allow for creating models on a higher level of abstraction 
than URLs - namely clusters of pages. Such models could be used for visualization of 
user access patterns (as in [7]) or, as noted in [3], to predict Web page entries on a different 
Web site but with similar structure. 



SEE. The way we developed SEE imposed on us the assumption that, before everything 
else, the general mechanism was needed. Now, when the module sorts links individually 
for each user, the lack of strong criteria has proved to be its main flaw. The criteria we have 
implemented only indicate how powerful SEE could be. They mainly test the percentage, 
on each rated page, of certain HTML tags, inside which are the keywords. The concept 
of semantics-driven criteria has accompanied the whole process of developing SEE. In 
other words, SEE could immensely benefit from clustering pages which cover the same 
topics. 

Another issue which SEE should deal with is the size of the model. SEE attempts to 
store information in pairs: the user and a given keyword. Hence the need for grouping 
users sharing common interests (in terms of criteria). SEE could also do with a way of 
clustering keywords that the users perceive as similar. 



7 Conclusion 



Adaptive web and personalizing Web servers are relatively young fields of computer 
science. In spite SIE is still an immature system, we hope it will help the scientists to 
test their ideas and develop new modules. Such a framework could prove very useful in 
social sciences, or in fields that include interaction with humans, as it is impossible to 
model their behavior in absolutely abstract way. We were not able to find any similar 
framework freely available on the Internet. We hope SIE will fill this gap and make future 
research easier. 
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Abstract. A challenge in supporting Wide Area Applications (WAA) is that of 
scalable performance management. Individual Latency Profiles (iLPs) were pro- 
posed in the literature to capture latency distributions experienced by clients when 
connecting to a server; it is a passive measurement made by client applications and 
is gathered on a continuous basis. In this paper, we propose a scalable technique for 
managing iLPs by aggregating them into aggregate Latency Profiles (aLPs). We 
use measures such as mutual information and correlation to compare the similarity 
of pairs of iLPs. 



1 Introduction 

Wide area applications (WAAs) utilize a WAN infrastructure, e.g., the Internet, to connect 
a federation of hundreds of servers, typically content providers, with tens of thousands 
of clients. Servers provide services that may range from simple downloads of digital 
content to complex Web services with multiple interchanges between client and server. 
It is expected that WAA must scale to millions of client and server pairs. As an example, 
consider a global name service such as the Handle protocol, an IETF/IRTF standard from 
CNRI- Corporation for National Research Initiatives [13]. Handle provides a namespace, 
a name resolution service, and protocols for digital object location and access. The Inter- 
national Digital Object Identifier (DOI) Foundation (www.doi.org) and the community 
of publishers utilize handles to facilitate the identification and exchange of intellectual 
property in the digital environment. It is expected that such applications must scale to 
tens of millions of Handles and thousands of content servers, representing the digital 
content managed by the publishing community, and large numbers of Handle clients. 

A significant challenge in deploying WAA is that of scalable performance manage- 
ment for large numbers of clients. The unpredictable behavior of a dynamic WAN [11, 
12] results in a wide variability in access latency (end-to-end delay). There has been 
extensive research in the networking literature to develop metrics and models to pre- 
dict latencies, including Internet distance and points of congestion [1,4, 9.1 1,12], There 
has been research on route aggregation based on IP prefixes exchanged via the Bor- 
der Gateway Protocol (BGP) and exploiting BGP information to monitor and predict 
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performance [5,7]. BGP routes expressed as paths via Autonomous Systems (ASes). 
However, an entire AS may not demonstrate homogeneous behavior, e.g., whenever it 
spans a large geographic area. Further, the effort to acquire knowledge of the BGP paths 
between different clients and servers may vary, since some clients and servers do not 
provide a looking glass service. Finally, while network topology is often a good predictor 
of latency, it may be the case that there is no available latency data for a closely matching 
client and server pair. Alternately, a client and server pair with similar BGP routing may 
not always be a good predictor of latency for the client and server of interest, e.g., if the 
two servers experience dissimilar workloads, or were associated with dissimilar points 
of congestion. Latency prediction models based on network characteristics alone would 
not be appropriate, or would not differentiate the cases described above. This too moti- 
vates the complementary need for a management tool and measures that do not rely on 
extensive (and sometime unavailable) knowledge of the network and its characteristics. 

In [10], we proposed latency profiles as a conceptual model to characterize the 
behavior of sources over a WAN. Latency profiles (LPs) are time-dependent latency 
distributions that capture the changing latencies clients experience when accessing a 
server; it is measured by client applications or middleware and is gathered passively and 
on a continuous basis. Latency profiles can be utilized as a WAA monitoring tool, to 
predict latencies that clients should expect in response to requests, using historical data 
and recurrent behavior patterns. However, in the presence of hundreds of servers and tens 
of thousands of clients, managing millions of latency profiles cannot scale. Therefore, we 
explore in this paper a method for aggregating latency profiles. We propose information 
theoretic and statistical measures such as mutual information and correlation to compare 
the similarity of pairs of iLPs. Individual latency profiles (iLPs) will be aggregated into 
an aggregate latency profile (aLP). A representative latency profile for this aggregate 
will then be maintained. Whenever a request for service arrives, a prediction will be 
based on the representative latency profile. Using aLPs allows us to discover aggregate 
performance patterns that would have been difficult to obtain using network topology 
and characteristics alone. We empirically show that there is a considerable amount of 
non-random associations between iLPs. While some of the strong associations can be 
explained based on physical network topology and characteristics, our experiment also 
shows that given a group of client and server ASes, with similar (overlap) of BGP routes, 
there may be a wide variation of the strength of non-random associations between pairs 
of iLPs. 



2 Wide Area Performance Monitoring 

Figure presents a WAA performance monitoring architecture. There are three types 
of nodes, namely clients, content servers, and performance monitors (PMs). Clients 
continuously download data from content servers and passively construct individal iLPs. 
PMs manage large collections of iLPs; this is done by aggregating iLPs into a smaller 
number of aLPs; PMs then manage some number of aLPs and the associated iLPs. Clients 
consult PMs to obtain a prediction. The scope of an aLP is depicted in Figure 1 by elipses, 
where each elipse contains clients and servers for each an iLP can be constructed. 

Suppose a latency prediction is requested for a pair (c, s) respresenting client c 
and server s. Suppose also the PM does not have an associated iLP, from the same 
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Fig. 1. WAA monitoring architecture based on performance profiles 

client AS of c to the same server AS of s, that can be directly used to predict latency. 
Alternatively, the system does not have sufficient resources to continuously maintain 
all profiles. Assume further that there exist an iLP\ associated with a client/server pair 
(ci , s) for a different client AS than that of c, but to the same server AS as of s. Similarly, 
there is an iLP 2 for client/server pair (c, Si) (same client AS as c and different server AS) 
and iLP$ for client/server pair (ci, si) (different client AS as c and different server AS as 
s). Now the PM can choose either iLP\ or iL If to make a prediction for the client/server 
pair (c, s ). It is also possible that there exists strong non-random associations between 
iLP\, iLIf and iLP^. In this case, the best estimate of access latency for (c, s ) is 
possibly obtained by aggregating iLPi, iLIf and iLP, 3 into an aggregate latency profile 
aLP, and choosing a representative profile. 

2.1 Individual and Aggregate Latency Profiles 

Given a client c, a server s, an object of size b, and a temporal domain T, an individual 
latency profile is a function iLP CtS : T x M —> U {T O}. iLP CtS (t , b) represents the 
end-to-end delay for a request from server s at time t, given as either a real number or 
using TO to represent a timeout. iLP C:S comes in two flavors, similar to [3]. One flavor 
measures time-to-first, which depends on factors such as workload at the server and 
size of the requested object. The other flavor measures time-to-last, which has a greater 
dependency on network bounds. Due to the stochastic nature of the network, iLP cs is 
clearly a random variable. 

An aggregate latency profile aL P, ij ‘ combines a set of n individual latency profiles 
iLP = {iLP Ci }” =1 . We construct an aLP by grouping iLP s with similar characteris- 
tics that are non-randomly associated with each other; this will ensure that the grouping 
will benefit the prediction ability of the aLP. For this grouping, we rely on information 
theoretic and statistical measures computed for the pair-wise association of iLPs. In par- 
ticular, we use mutual information [2], and correlation [ 8 ]. A higher mutual information 
between two iLP s means that those iLP s are non-randomly associated. Conversely, 
a mutual information of zero means that the join distribution of iLP s holds no more 
information than their individual distributions. A higher correlation between two iLPs 
can also indicate that those iLP s are non-randomly associated. In general, there is no 
straightforward relationship between correlation and MI [ 6 ]. While correlation captures 
linear dependence, mutual information is a general dependence measure. 

After constructing an aLP from a set of non-random associated iLPs , we can im- 
prove the prediction of an iLP by using observations of other iLPs in the aLP. Recall 
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that using an observation of a random variable Y which is related to a random variable X 
in some way, e.g., Y is non-randomly correlated with X, an optimal mean-square-error 
estimator of X given Y is the conditional expectation of X given Y, E (X\Y) [8]. We 
use conditional expectation to utilize the meaningful relationships within an aLP in 
order to improve latency prediction. 
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Fig. 2. Distribution of Average Relative Estimation Error 



3 Experiments 

In this section we report on part of our experiences with constructing aLP s. The experi- 
mental data was collected over the CNRI Handle testbed [ 13], - an emerging IETF/IRTF 
standard that provides a global name service for use over WANs. We gathered data from 
November to December 2002. The data is typically PDF files that are reachable via Han- 
dle resolution. We report on the performance of 22 clients (2 each on 1 1 client ASes) 
accessing 10 servers, yielding 220 iLPs. We explored two approaches for grouping iLPs 
in aLPs, namely using mutual information and using correlation. 

We group strongly related iLPs in aLP. We applied conditional expectation to 
esimate individual latencies using observations of latencies from a representative iLPs 
within one aLP. All our aLPs in this experiment consisted of two iLPs. For each aLP 
{iLPi, iLPj} we esitmated latencies of iLPi using observations of iLPj, i.e., we 
choose iLPj as a representative profile. 

We computed the average relative estimation errors for all iLP pairs (aLPs) consid- 
ered in our experiment. Relative estimation error is defined as abs(x — x es t)/x, where 
x and x es t are actual and estimated latencies correspondingly. For each aLP {iLPi, 
iLPj } we average the relative errors of estimation of all individual latencies from iLPi. 
Figure 2 plots the distribution of the average relative estimation error. We observe that 
variability of the relative error is considerable. Figure 2 shows that major part of esti- 
mation errors (about 9000 estimations) is in a good range of [0,1]. However, more then 
1000 estimation errors are large (above 3), and as we see from Figure 2, they can be as 
much as 75. Meanwhile, from our experiments we found that practically all of the large 
estimation errors spread over areas of low MI ( < 0.4 ) and low correlation ( < 0.2 ). 

We observed that using MI and correlation to construct aLPs does not always guar- 
anty the best latency estimation, but it helps to maintain good estimation quality. More- 
over, avoiding non-related representative iLP s effciently eliminates large estimation 
errors. We conclude that aggregating non-randomly associated latency profiles can prac- 
tically assist in wide area performance monitoring. 
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4 Conclusion 

We have presented the concept of an aggregate latency profiles as a scalable method- 
ology for utilizing latency profiles. Mutual information and correlations are compared 
in their ability to explore useful aggregate latency profiles. Our experiments show that 
in general correlation serves better is generating aggregate latency profiles and in pre- 
dicting latencies. We plan on implementing our methods in a prototype, allowing the 
generation of aggregate latency profiles and testing them out in retrieving documents 
based on handle information. We are going to use more advanced prediction techniques 
such as Neural Networks and Web Prediction Tool [14], to fully utilize prediction power 
of aggregate latency profiles. 
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1 Introduction 

The rapid growth of uncacheable content over the HTTP protocol [1,2] necessitates the 
further investigation and exploitation of its properties. In this paper, we intend to answer 
these following questions: Among the huge HTTP content delivered on the Internet, 
which parts are cacheable and which parts are uncacheable? What are their charac- 
teristics, especially for uncacheable content? Is there any difference among different 
uncacheable HTTP content? Is there any cacheable possibility for these conventional 
uncacheable content? How about the cacheability of personalized content? Is there some 
relationship between uncacheable content and HTTP persistent connection? 

To answer these questions, we sniffed and analyzed all inbound and outbound HTTP 
traffic on all possible TCP ports at a medium-size education institution. By analyzing an 
one-day trace, we observed the following: (1) uncacheable data have dominated today’s 
HTTP traffic and multimedia type content transferred by P2P applications (35.5% ) and 
graphic type (jpeg and gif) content (17.3% ) are top two in the uncacheable HTTP 
objects; (2) compared with cacheable content, uncacheable content consumes more 
server-processing time, but due to network latency, the client-perceived response time 
tends to be close to that of cacheable content; (3) on average, uncacheable content has 
an larger object size than that of cacheable objects (13 K bytes vs. 7K bytes); (4) clients 
accessing personalized content and servers providing personalized content are more 
concentrated than general clients and server groups, while the total online personalized 
content occupies only a smaller percentage (less than 10%). far below than previous 
observations from [2]; (5) a considerable (50%) portion of HTTP objects have their TTL 
values equal to zero. Among these objects, we observed that the URL aliasing contributes 
20% of total requests; (6) P2P traffic is increasingly scattered among multiple ports (only 
13% on default ports for KaZaA traffic), which is a big challenge for deployment of P2P 
traffic caching. Several implications could be derived based on above observations: 
(1) a considerable portion of uncacheable HTTP content is cacheable; (2) domination 
of network latency factor motivates the moving of functionality for uncacheable HTTP 
content generation to the edge of networks; (3) prefetching for personalized Web content 
is promising because of the concentrated popularity of clients and servers; (4) convinced 
by the observed P2P request popularity, we believe that the content-based caching is 
significant to the ever-increasing P2P traffic; (5) exploiting URL-alias is a promising 
direction to improve cacheability of uncacheable content. 
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2 Analysis Results 

2.1 Trace Collection 

We collected one-day period (12:00 pm, Mar 18 -12:00pm Mar 19, 2003) HTTP traffic, 
rebuilt and investigated the contained content. To capture all possible HTTP traffic, 
TCP packets on all ports were sniffed. We have developed WebTACT, an Web Traffic 
Analysis and Characterize Tool, to extract the complete HTTP information, including 
both header and content. Analysis results and observations are depicted in the following. 
Due to space limitation, we present high level results only, more detailed data and analysis 
can be found in the technical report version of this paper [3], 

From the viewpoint of proxy caching, generally HTTP object could be broadly 
categorized as uncacheable content or cacheable content. The cacheable HTTP objects 
refer to those infrequently changed HTTP objects (also known as static HTTP content), 
and the uncacheable HTTP content could be further classified into uncacheable subtypes 
one to seven respectively: NonGet, DynGen, Pragma, CacheCtl, Personalized, 
AbnormalStatus and ZeroTTL, based on the HTTP protocol specification [4], 



2.2 High Level Characteristics 

Table 1 lists the high level statistics for both cacheable and uncacheable content. For each 
content type, we detail them in different traffic directions. The inbound traffic means that 
the response objects are targeted to clients inside the campus, while the outbound traffic 
means that the response objects are targeted to clients outside the campus. The total 
distinct client number inside the campus is 9,053, and that outside campus is 93,250. 
The total server (host providing HTTP content) number inside the campus is 1,930, and 
that outside the campus is 1 14,416. 

From Table 1, we can see that, the captured-reconstructed gross HTTP traffic (in- 
clude HTTP headers and bodies) is around 117.5 GB. For the total objects size (or the 
total size of transferred HTTP response objects), 1 the uncacheable content outnumbers 
the cacheable content (72 GB vs. 2.7 GB). The servers that providing uncacheable con- 
tent outnumber those providing cacheable content (116, 149 vs. 7,5 1 8), while the clients 
accessing dynamic content also largely outnumber the clients accessing cacheable con- 
tent) 10 1,971 vs. 14,674). These data exemplify that the uncacheable content dominates 
today’s HTTP traffic and the necessity of efficient delivery of uncacheable content. 
The majority of HTTP uncacheable traffic is multimedia audio/video type. This is be- 
cause that: (1) the huge volume of P2P (KaZaA) application traffic focuses mainly on 
multimedia file exchange; (2) 99.6% KaZaA HTTP objects are categorized into un- 
cacheable type by our analyzer. Comparing our data with previous results in [2], we 
observe an increase in the uncacheable request/response for image (gif and jpeg) con- 
tent type, and a decrease in text (html and plain) content type. The possible reason is 
the widely acceptance of cache busting technologies [4], The multimedia type objects 
(video/x-msvideo, video/mpeg and audio/mpeg), which contribute to a large per- 
centage of total bytes and a small percentage of total number of responses, implies a 
larger average size of these kinds of objects. 

1 Hereafter, all the traffic mentioned in this paper are referring to total objects size. 
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Table 1. High-level statistics of HTTP traffic. 



Type 


Cacheable 


Uncacheable 1 


HTTP Traffic Direction 


Inbound 


Outbound 


Inbound 


Outbound 


# of Servers 


7,345 


173 


114,221 


1928 


# of Clients 


7,007 


7,667 


9,050 


92,921 


Total Gross Traffic(MB) 


2,177 


875 


66,278 


51,017 


Total Object Size (MB) 


1,917 


825 


42,209 


31,619 


# of Requests 


309,616 


73,662 


6,688,378 


2,181,252 



2.3 Detailed Characteristics of Uncacheable HTTP Content 

Uncacheable Content Breakdown. Table 2 lists the absolute numbers for each of the 
seven uncacheable subtypes, by their total object size and request/response number. The 
“mixed” type means the uncacheable subtype is a combination of this subtype and at 
least one other uncacheable subtype, while the “pure” type means the request belongs to 
this subtype only. We find that the personalized objects (subtype 5) occupy less than 
ten percent of all uncacheable content in terms of both bytes and number of requests, not 
as large as previous observation. We do not know the exact reason for this low percentage 
of personalized HTTP objects. A distinguished portion, ZeroTTL (subtype 7), implies 
a promising probability of caching performance improvement that we will give more 
detail analysis later. 



Table 2. Detail breakdown for all seven uncacheable subtypes. 



Subtype 


ZeroTTL 


AbnormalStatus 


Personalized 


CacheCtl 


Pragma 


DynGen 


NonGet 


pure requests 


4,413,886 


2,067,218 


71,634 


104,186 


80,947 


1,274,334 


69,767 


pure size(MB) 


58,182 


1,167 


541 


974 


2926 


6,493 


276 


mixed requests 


0 


574,645 


92,989 


402,960 


232,048 


476,420 


80,710 


mixed size(MB) 


0 


348 


398 


2,360 


72 


2.819 


352 



Response Time and Breakdown for Uncacheable Content. For further analysis, we 
first want to know, whether the cacheability of objects affects their response time, on 
both server side (processing time) and client side (latency). Because of the sniffing point 
location of our study, we could assume that for the inbound traffic, the response time 
is close to client-perceived latency, and for the outbound traffic, the response time is 
close to server-processing time. For the inbound HTTP traffic, the difference between 
the response time of uncacheable and cacheable objects is not so much. This implies 
that the time difference caused by dynamic/static content generation has been blurred by 
the network latency on their route. For the outbound HTTP traffic, there is a difference 
between the response time of uncacheable and cacheable content, this is probably caused 
by the time necessary to dynamically generate the uncacheable content. The response 
times for different dynamic types do not show much difference, especially for inbound 
traffic. These phenomena show the need to migrating the dynamic generation functions 
to the network edge. 

Object Size Distribution. The distribution of object size is also an interesting topic, 
especially when HTTP objects are classified into two major classes: cacheable and 
uncacheable. Intuitively we believe that, on average, uncacheable object size is smaller 
than cacheable size, but our analysis gives contrary result. 90% of cacheable objects 
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smaller than 14,860 bytes, while same percentage uncacheable objects are smaller than 
18,694 bytes. The average size is 7K bytes (cacheable) vs. 13K bytes (uncacheable). 

Amazingly, the largest HTTP object size we observed is 252 M bytes for uncacheable 
object and 12M bytes for cacheable objects. These numbers are much smaller than those 
appear in [5], The possible reasons are: (1) our data collecting period is relatively short 
(24 hours vs. 9 days data collecting period [5]); (2) the object size is calculated based on 
the bytes on the wire, instead of the HTTP headers. As more and more applications (e.g., 
KaZaA) adopt parallel downloading or other segment-based content delivery techniques, 
supported by the HTTP protocol, we believe the size of individual HTTP objects will 
not be larger any more. So the real reconstructed (fragmented) objects reflecting only a 
fraction of total size is a reasonable explanation. 

Is Uncacheable Content Really Uncacheable? Although the object composition tech- 
nique, such as ESI , has been proposed, in this paper we are looking for URL-alias derived 
cacheable possibility. Totally, there are 4,413,886 objects belonging to ZeroTTL sub- 
type, Among these we observe that many different URLs share the identical content 
digest. This is caused by the phenomenon called “URL-alias” [6], Our analysis show 
that the number of requests targeting the top 1000 (less than 0.1% of total distinct digest 
value) rank digest value count for 18% of total number of requests. This observation 
reveals an opportunity for the future Web cache improvement if certain protocol could 
be designed to deal with ZeroTTL objects based on their digest value, rather than on 
their URLs only. 

Client/Server Popularity. We assume that the personalized Web content would be more 
client/server-specific than general content, due to its “personalized” property and the 
analysis results do verify our assumption. Top 1% of clients that accessing personalized 
content bring about 20% of the total personalized content requests. However, unlike 
previous observations [2], we find that clients interested in personalized content only 
occupy 2% of the total client population. Some clients are much more likely to access 
personalized Web content. These clients are some public-access computers, located at 
public area like student dormitories, for students check updated personalized information 
like email or personal account on e-commerce Web sites. We also find that personalized 
content is provided by 1% of the total servers, and servers providing personalized content 
are also more concentrated than server providing general content. Top 1% of servers 
that provide personalized content handle 85% of the requests for personalized content 
requests. There are “hot” personalized Web servers and the top 30 of the servers contribute 
95% of the total requests among the top 100 servers providing personalized content. 

Object Popularity. Due to the personalized property, personalized HTTP objects might 
not be more concentrated than general objects and this is proved by our analysis. All these 
observations strongly suggest that personalized prefetching could be used to reduce the 
client perceived latency and network bandwidth equipment. 

Persistent Connection vs. Uncacheable HTTP Objects. In our reconstructed HTTP 
data, there are totally 669,958 persistent connection sessions, consists of 3,411,741 
HTTP request/response pairs. On average, a persistent session consists of 5.09 pairs. We 
have supposed that the uncacheable content would have some distribution patterns among 
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multiple pairs within one persistent connection. One reasonable assumption is that, for 
a persistent HTTP connection, maybe the first object is an uncacheable dynamically 
generated page template, followed with embedded cacheable objects like graphics. But 
our analysis results deny this distribution pattern proposition and conclude that most of 
the uncacheable content does not appear in persistent connections. 

Peer-to-Peer Traffic Analysis. We reconstructed all http-based P2P traffic by captur- 
ing all TCP traffic, instead of sniffing only some specific default ports (e.g., 1214 for 
KaZaA, 6346 and 6347 for Gnutella, used by previous work [5]). Generally, as observed 
earlier in this Section, P2P traffic contributes to a large portion of total HTTP traffic. 
For Gnutella type P2P applications, we aggregate several found Gnutella client appli- 
cations (e.g., LimeWire, BearShare, Shareaza etc.) into a whole Gnutella division. 
For KaZaA type data, only the traffic from KaZaA client is calculated. The total HTTP 
object size transferred by Gnutella applications is 1,110,383,667 bytes, while that by 
KaZaA application is 26,659,686,069 bytes. The total object size transferred by P2P 
applications occupies 33.8% of the total observed HTTP object size while the corre- 
sponding percentage is over 75% in [5]’s work. With the evolution of P2P applications, 
P2P traffic ports are more distributed than before. For example, only 13% of KaZaA 
traffic is through its default port 1214. This phenomenon strongly implies the emergence 
of new mechanisms dealing with P2P traffic spreading on different TCP ports. 

3 Summary 

Implied by characteristics analysis of uncacheable HTTP traffic, we propose four promis- 
ing directions to improve caching and content delivery of uncacheable HTTP content: 
first, pushing the functionality of uncacheable content generation to the network edges; 
second, applying the access pattern feature to prefetching schemes; third, implementing 
an efficient content-based P2P traffic caching; Finally combining content-based approach 
into current cache to exploit the prevailing URL-alias phenomenon. 
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Abstract. In recent years, more and more publications and material for studying 
and teaching, e.g., for Web-based teaching (WBT), appear "online" and digital li- 
braries are built to manage such publications and online materials. Therefore, the 
most important concerns are related to the problem of durable, sustained storage 
and the management of content together with its metadata existing in heteroge- 
neous styles and formats. In this paper, we present specific techniques, their use 
to support metadata-based catalog services deploying XML, and finally derive the 
concepts of a suitable component-based architecture. 



1 Introduction 

More and more e-learning material is produced for online services. In general, it is 
integrated into portal systems [1] and can be used via the Web. However, searching 
online material is not easy. Therefore, digital libraries and online catalog services have 
been built. First of all, information about online material has to be collected and made 
queryable. For this purpose, a metadata repository with search (full text) or query (struc- 
tured) facilities has to be made available. Furthermore, online catalog services should 
deploy state-of-the-art frameworks and architectural concepts to deal with scalability 
requirements. Particularly with regard to the very central task of search and query sup- 
port, scalability is essential. Finally, online catalog services stand for more than simple 
search engines. In fact, they provide search and query support, but additional sophisti- 
cated information about the query result is needed, which is presented in more detail in 
the next sections. 



2 Requirements 

To provide an online catalog, it first has to be filled with useful material. In an initial 
step, resources have to be found and identified. If acceptable, they have to be indexed 
and reviewed. All of these steps should be supported by an appropriate management 
system, for which the underlying process is illustrated in Fig. 1. The principal task 
consists of efficient query support. Short response time, high throughput, low resource 
consumption, integration of full-text search and structured queries, as well as support 
of highly concurrent user queries are requirements summarized by the term “efficient 
query support" . In addition, various kinds of statistics - related to the original queries, the 
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most frequent subjects, the number of hits, or the variance of a distinct value distribution 
— are useful, e.g., to enable a comprehensive personalization concept, i.e., to provide 
navigational hints or help on choosing a good sorting or ranking method. 

The list of requirements 
of our project made it neces- 
sary to define a special set of 




Fig. 1 . Basic Indexing Process 



metadata attributes tailored 
to its special needs. For this 



reason, we included special metadata attributes for online resources and peer reviews to 



deal with didactical issues and issues at law. In fact, for various reasons our project [2] has 



chosen Dublin Core and its meanwhile obsolete qualifier approach as a basis for defining 



a hierarchical scheme. This data model is aware of elements and attributes and a number 



of data types. It offers the specification of single-value elements, ordered lists, or un- 
ordered sets of elements. Metadata records are in relationship with other records. In this 
way, complex tree-like structures may result. A learning resource record, for example, 
may be in relationship with a user comment, a peer review, or again a learning resource, 
e.g., if the material occurs in different formats (like PostScript or the widespread PDF 
format). Deploying XML as formal representation makes it possible to describe the 
transformation of our metadata set representation into RDF or OAI-like (Open Archive 
Initiative) structures in a very formal way to easily support interoperability with other sys- 
tems, e.g., within library consortia. The XML-based representation enables two distinct 
ways of processing the underlying data: element-based (fine-grained) or document-based 
(coarse-grained). Hence, especially in enterprise applications a document-centric, i.e., 
coarse-grained approach should be preferred. Therefore, the process model including 
definition, query, and retrieval of data is XML-based. 



3 Basics 

With the advent of object orientation (OO), many old software engineering problems 
could be solved, but some problems still remain. OO and the corresponding languages 
only support fine-grained concepts. Hence, scalability in large or data-intensive applica- 
tions could become a problem. For this reason, components have emerged. Components 
are a coherent package of software that can be independently developed and delivered 
as a unit and that offers interfaces by which it can be connected with other compo- 
nents to compose a larger system. Its physical implementation is held in one place and 
components will often be larger-grained comparing to the traditional concept of ob- 
jects. Because the standard Web-communication protocol HTTP is a request-response 
protocol, individual requests of Web-based applications are treated independently. For 
this reason, state management truly becomes an issue at the application layer. Some- 
times it is advised to push state management completely back into the database tier [3]. 
With stateless implementations, resource sharing is easy, because they are exclusively 
used only by one component during a single method execution. Apparently, a central- 
ized storage for component state seems to be very practical. The alteration from object 
orientation to component-based models is accompanied by a shift in paradigm related 
to the corresponding architectures. Traditional client/server architectures are displaced 
by enterprise application architectures. Even though enterprise application architectures 
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could be considered as distributed n-tier client/server architectures, they are derived 
by the methodical appliance of engineering techniques and also by taking business ap- 
plication requirements into account. One of the most important techniques are design 
patterns. 



4 Architecture Derivation 



The main focus of our approach lies on the separation of concerns. Therefore, distinct 
tasks have to be identified and have to be separated. First of all, we have separated 
the data (i.e., the learning object) from its metadata and the metadata from its related 
metadata (so-called administrative metadata). Furthermore, beside the above separation 
of data and metadata we have made a strict distinction between three different main 
tasks: query processing, result set processing, and XML document processing. 

Each of these tasks is realized by a separate component which places us in a 
promising position to achieve better scalability. As depicted in Fig. 2, the separation 

of business and application 
(i.e., framework-specific) 
logic is essential for enter- 
prise programming. In the 
bottom tier (enterprise in- 
formation system tier) data 
is managed. Unstructured 
data (or learning objects) 
are managed and indexed by 
a freely-available full-text 
indexing system, whereas 
structured metadata is 
stored in a relational 
database system (RDBS). 
Therefore, we need a 
transformation between 
the XML model and the 
relational data model with 
simultaneous consideration 
of the integration of the 
full-text index. Support for this transformation can be found in the XML/R mapping- 
support layer located at the server-side processing tier. It builds the foundation of the 
higher-level server-sided components which include services for meta model access, 
domain-value queries, or configuration management. 

In addition, the server-side processing tier provides components for query processing 
which is separated into three distinct steps. In the first step, a client query against an XML 
schema is transformed into an equivalent SQL query by the query processor which pro- 
vides methods to execute or refine queries and expects a descriptive query specification. 
The result of such a query is cached and processed by an additional component (result 
set processor). The XML documents qualified for a query could be modified by the third 




Fig. 2. Architectural overview 
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component (XML processor), if needed. These components act like a controller in ac- 
cordance with the MVC pattern [4], Thus, they mediate between the underlying data tier 
containing the models and the presentational views. The business interface implemented 
by its corresponding controller represents the distinct component interface and helps to 
abstract from the particular application-specific or frameworkspecific implementation 
details. The implementation of this interface realizes coarsegrained business methods by 
composition of less complex methods and is conceptually equivalent to SUN’s Session 
Facade pattern [4], 

The server-side presentation tier is located above the server-side processing tier. 
Web modules for supporting Web-based applications, for instance the Web-based search 
engine component, can also be found here as well as Web service connectors, which 
provide interoperability to special clients (e.g., in the case of OAI support, deployment 
of external indexing, or harvesting clients). Beside thin clients (i.e., user agents like 
Web browsers) fat clients for maintenance and indexing tasks are supported, too. Again, 
to abstract from particular application-specific or framework-specific implementation 
details, the business delegation pattern is deployed where all these details could be 
encapsulated. The business delegation pattern implementation builds a single point of 
communication between Web modules located in the server-side presentation tier and 
the various controllers in the server-side processing tier. 

As a consequence of the above mentioned separation of concerns, query processing 
is separated from result set processing. This is achieved by caching the result set and 
providing a result set handle for identification purpose. But caching is also necessary to 
accomplish some reordering of the result set, because the full-text search engine does not 
support comprehensive sorting capabilities. However, caching even offers some more 
advantages. To determine the number of hits, the most frequently used terms or subjects, 
or to make helpful hints to determine appropriate search criteria by building the variance 
of distinct columns, it is not necessary to rerun the entire, typically very expensive, 
query. Beyond these so-called metaqueries it is feasible to use the cached results for 
query refinement, for reuse during processing of subsequent queries and for provision 
of a so-called stateless cursor. It is not feasible to hold a database cursor open all the 
time a user navigates through the result set pages, because open database connections 
are too expensive. In the case of stateful components, there exists a component instance 
for each single client request. An implementation has to be realized in a stateless manner 
and state can be managed by a distinct component or a RDBS. In our approach, the result 
set of a client query is stored into a separated (temporary) table of a RDBS together with 
additional administrative data (e.g., cursor position). Hence, concurrent queries and locks 
are no problem any longer and every subsequent query transformation or refinement of 
the result set could be locally executed in the cache. 



5 Scalability 

The trichotomy of the overall query process comes along with a very positive impact. 
By deploying such an architecture, we can cope with the typical workloads of digital 
library usage patterns and achieve better scalability. Typical workloads rely on the as- 
sumption that a user tries out some queries and refines a suitable query result by selecting 
a navigational hint to narrow the result set or by using one of the sort methods offered. 
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Additionally, information about the current query result should be presented to the user, 
e.g., the total amount of hits, how many documents are classified by a distinct subject, or 
information about the most frequent keywords. Though, evaluating such information is 
expensive. In most cases, a query has to be repeated more than one time to build suitable 
aggregated information about distinct columns of a result set. Regarding our require- 
ments, a query has to be executed at least four times to aggregate all of the information 
needed. Incorporating the cache to evaluate, the above mentioned, metaqueries allows to 
specify very simple and inexpensive queries which get along with a few simple joins and 
which relieve the underlying RDBS. Furthermore, our approach yields another benefit 
regarding sort support. Consecutive queries differing solely in the sort criteria applied 
could be answered without the necessity to repeat the query. 

In addition to caching operational data of a distinct user query, conversational state 
information is stored, too. For this reason, a stateless component architecture becomes 
feasible. The state is managed outside of the component and could be restored, before 
processing is continued regardless where the component resides, i.e., in the case of 
distributed processing, each node could answer any request. The query engine itself is 
implemented in a stateless manner and can be distributed, too. Result set processing is 
done by a separate component which operates on top of the cache. For this reason, initial 
queries are processed by the query engine and subsequent queries are performed locally. 
As a consequence, subsequent queries do not stress the underlying database system. 



6 Conclusions and Outlook 

In this paper, a scalable component-based architecture for online catalog services has 
been presented that can be used in the domain of digital libraries. In contrast to search 
engine services (e.g., Google or Altavista), appropriate products (e.g., ASPseek), direc- 
tory services (e.g., Yahoo or DMOZ), or other digital library systems (e.g., MyCoRe), 
our approach is able to cope with a more comprehensive set of metadata attributes and 
uses standardized techniques as well as a component-based framework (J2EE). It is very 
flexible concerning the underlying metadata schema. In addition, automatic indexing ser- 
vices are integrated in our system. Thus, interoperability, scalability, process-oriented 
metadata management, and the integration of full-text search and structured queries 
are the outstanding characteristics of our approach. MyCoRe can be deployed for the 
development of Digital Library and archive solutions. Adjustability, extensibility, and 
open interfaces are fundamental design premises of MyCoRe, but, at the moment, it is 
dedicated to only a single database management system [5], Therefore, the innovative 
aspect of our approach could be found in the segmentation of the query process. As a 
consequence, sophisticated caching becomes feasible. It is not necessary any longer to 
rerun expensive queries which could additionally include unstructured search to answer 
metaqueries and to accomplish the requirements of personalization. The concepts pre- 
sented have been validated by realizing a prototype system 1 . Furthermore, in-memory 
cache management at the middle-tier may be considered which is similar to the approach 
described in [6]. 



1 www.akleon.de (at the moment only available in German) 
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Abstract. The next version of XHTML is at work-in-progress stage 
in the World Wide Web Consortium. It adds a lot of features to the 
most used content language of the Web. The most notable change is the 
addition of XForms, the next generation WWW forms language. This 
paper describes the XHTML 2.0 specification and an XML user agent 
implementation for it. The new features of the language are discussed 
both from the author’s and the user agent manufacturer’s point of view. 
In addition, it describes a case study, which takes advantage of the new 
features. 



1 Introduction 

HyperText Markup Language (HTML) [1] is one of the greatest factors to the 
success of the World Wide Web (WWW). It provides an easy way for authors to 
create content to WWW. Originally, HTML was designed to describe structure 
of the document. Later, a lot of presentational features were added into it. 

To bring HTML back to its origin, it was redefined as an Extensible Markup 
Language (XML) [2]. First versions of XHTML were just reformulation of HTML 
4, but the newest version, XHTML 2.0 [3], which is at work-in-progress stage 
in the World Wide Web Consortium (W3C), is no longer backward compatible 
with the earlier versions. As a starting point to a design of XHTML 2.0 has 
been experiences and problems of earlier web technologies, especially HTML. 
The intention is to make XHTML 2.0 easy to adopt for authors and to match to 
original purpose of HTML (i.e., describe the structure of hypertext documents). 
For instance, XHTML 2.0 removes all presentation elements from HTML and 
subordinates all presentation to stylesheets. In addition, a lot of functionality 
has been added to XHTML 2.0. The objective is to reduce the use of scripting 
languages within XHTML documents. Most common scripts have been replaced 
by functional elements. 

The success of the WWW is also largely based on interactive services such 
as search engines, online banking, and e-commerce. A key technology used in 
interactive Web applications is HTML forms. However, requirements for these 
services have steadily increased since the advent of the technology. Today’s high- 
end forms use complex client-side ECMAScript [4] programming to achieve form 
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field validation and simple computations (or bounce the form back and forth to 
the server). Heavy use of scripting inevitably leads to low maintainability and 
accessibility. [5] 

The HTML forms has been replaced by XForms in XHTML 2.0. That re- 
moves need for scripts from the forms and separates model and presentation of 
a form. Consequently completing the design principles of XHTML 2.0. 

The transition to XHTML 2.0 will be more difficult than transitions between 
earlier HTML versions. XForms [6] will be the biggest change in XHTML 2.0, 
but there are also other changes, which are not backward compatible [7]. In 
this paper, we have assessed the impacts of the transition to both user agent 
developers and authors. 

The paper describes XHTML 2.0 specification and an XML user agent im- 
plementation for it. Next section introduces XHTML and XForms modules of 
XHTML 2.0 and their integration. Sections 3, 4, and 5 discuss implementations 
of X-Smiles XML browser, XHTML module, and XForms module, respectively. 
A case study is described in section 6, while discussion about the transition to 
XHTML 2.0 is in section 7. Finally, section 8 gives the conclusions. 

2 XHTML 2.0 

XHTML 2.0, which is a W3C Working Draft, is a markup language, which 
describes structure of a document. It does not represent document’s layout. 
XHTML 2.0 is successor of the earlier HTML languages, but it is not backward 
compatible with them. 

XHTML 2.0 is supposed to be as generic XML as possible. Since XHTML 
2.0 is an XML language, it can be combined with other XML languages and use 
their features. Only special-purpose XHTML features were included in XHTML 
2.0. These are, for example, elements for images, tables, menus, etc. In addition, 
XHTML 2.0 uses facilities like XForms, XML Events [8], and XML Base [9]. In 
addition, layout of the document is defined by Cascading Style Sheets (CSS). 
All the presentational elements, such as font are removed from XHTML 2.0. 

The separation of content and presentation has many advantages. Layout of 
the document can be easily changed, layout can be specified for different devices, 
documents are smaller, and user agents are easier to implement. For instance, 
from the user agent vendor’s point of view the user agent’s components can 
be separated and they can be smaller, while the author can more easily reuse 
styling. This also leads to better accessibility because it is easy to change the 
styling of a document without touching the contents. 

Compared to earlier versions, XHTML 2.0 has better ability to make docu- 
ments more structured. Content is intended to be divided into sections, which 
all can have titles and subsections. Sections within sections can nest indefinitely 
deep. As before, content in sections consist of paragraphs, tables, list, etc. 

One aim of XHTML 2.0 is to reduce scripting from the documents. Many 
elements have now some kind of functionality as a default. That reduces device 
dependency and eases authors work, because they do not have care about script 
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languages. It is still possible to use scripting via the Document Object Model 
(DOM) interfaces. Other benefits of moving from scripts to declarative languages 
are improved accessibility and device-independence. This results from the fact 
that it is easier to use automated tools to process declarative markup than 
scripted documents. 

XHTML languages are divided into modules. XHTML 2.0 contains all the 
modules defined in XHTML Modularization 1.0 [10]. Although, content of the 
modules have been changed a bit. In addition, it uses modules from XForms, 
XML Events, and Ruby. XForms module is discussed in more detail in next 
subsection. XML Events provides an uniform way to integrate DOM Level 2 
event interfaces with event listeners and handlers. Ruby module is used to add 
short annotations to the text. Usually it is used as pronunciation instructions 
with eastern languages. The Ruby module is out of scope of this paper. 

The biggest changes in XHTML module are the addition of structural and 
functional elements. Functional elements had to be realized by scripts earlier. 
Now, there are elements, which have some default functionality. For instance, 
menus are common on the web pages. For that, there is navigation list in XHTML 
2.0 [11]. Navigation list is like normal list, but only selected part of the list is 
shown at a time. The part can be selected for instance by moving cursor on it. It 
is possible to style such a list as a drop-down menu with CSS, as shown in Fig. 
1. XForms specification also contains lot of features, whose intent is to replace 
scripting. 
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Fig. 1 . Navigation list styled as a drop-down menu 



Also, the use of attributes has changed. Attributes can be used more gen- 
erally than before. For example, href attribute can be added to any element. 
That makes the a element useless. In addition, all the elements can have sre 
attribute. That enables addition of external sources to the element. Element’s 
normal content is shown, if source is not available or cannot be shown in the 
device in question. 
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2.1 XForms 

HTML forms has many well-known shortcomings. It has no separation of data 
and presentation. Building anything more than a simple form requires excessive 
amounts of scripting, which is hard to implement and maintain. Another often- 
used approach is to send the form back and forth between the browser and the 
server, which leads to great amount of round-trips and reduced maintenance of 
the forms. 

XForms 1.0 Recommendation is the next-generation Web forms language, 
designed by the W3C. It solves some of the problems found in the HTML forms 
by separating the purpose from the presentation and using declarative ways to 
describe the most common operations in form-based applications. It can use any 
XML grammar to describe the content of the form (the instance data). Thus, 
it is also possible to create generic editors for different XML grammars with 
XForms. 

XForms separates the form into three main layers: Model, Instance Data, and 
User Interface (cf. Fig. 2 (a)). The Model layer includes the XML Schema and 
constraints. With the schema, it is possible to define the structure and the data 
types of the instance data, but XForms can also be authored without a schema. 
The schema can be processed via a normal XML Schema processor as shown in 
Fig. 2 (a). 
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<input> 
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(b) </input> 
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Fig. 2. (a) XForms layers and (b) the pseudo-class :: value inside a form control 



It is possible to define dynamic relations (constraints) between parts of the 
instance data. These relations are defined using XPath expressions. Examples 
of dynamic relations include inter-field calculations and constraints, which dy- 
namically set the state of an item in instance data to read-only or required. The 
calculation engine interacts with the instance data and it resolves and computes 
these relations dynamically while the user interacts with the form [5]. 

The User Interface is also bound to the instance data using XPath expres- 
sions. The choice of form controls covers the range of typical GUI widgets (al- 
though they are defined in higher level terms, such as selectl instead of menu). 
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Some form controls also adapt to the underlying XML Schema data type. For 
instance, an input control bound to a date data type will show a calendar picker 
instead of a text field. XForms also hosts a collection of dynamic user interface 
features, such as displaying and dynamically modifying collections of repeating 
items and switching parts of the user interface on and off. [12] 

XForms exposes a lot of the processing to the author via DOM events. For 
instance, there are events that are thrown when the instance data becomes in- 
valid, read only, required, etc. The author can catch these events declaratively 
using XML Events. The counterpart to XML Events are XForms actions, which 
can be executed when a certain event condition is met. This way it is possible 
to define, in a declarative fashion, certain actions to happen when the user in- 
teracts with the form. For instance, the form could be submitted partially when 
the user selects a certain item from a selection list. 

2.2 Integration of XForms and XHTML 

XForms is not a self-standing document type, i.e., it needs a host language 
to define the document’s master layout. XHTML is a natural choice as a host 
language in Web context. The current version of the XHTML 2.0 Working Draft 
includes XForms as the forms module. 

Since XHTML relies heavily on CSS2 layout, it is important that XForms 
also integrates to this model well. For instance, it uses the notions of pseudo- 
class, and pseudo-element from CSS. For instance, the value” pseudo-element 
is used to define the appearance of the UI widget itself (it would otherwise 
be hard to style just the widget, since in XForms, the form control element 
also encapsulates a label element). Fig. 2 (b) shows this; the first code snippet 
shows what the author writes to create a form control. It contains only the input 
element and a label child. To give the author better CSS control of the rendering, 
the specification describes an additional CSS pseudo-element, called ”::value”. 
The element input and label should be laid out using the CSS rules (such as 
display: block or display: inline) and the additional pseudo-element :: value is used 
to define style just for that part of the control, which the user can interact with; 
in this case an input box. 

Other pseudo-elements defined non-normatively in the XForms specification 
are ::repeat-item and : :repeat-index. The ::repeat-item pseudo-element is used to 
style all the items in a repeating collection, while :: repeat-index pseudo-element 
is used to style the currently selected item. 

3 The X-Smiles XML Browser 

X-Smiles is an open source XML browser developed at the Helsinki University 
of Technology. The authors of this paper have implemented XHTML 2.0 and 
XForms support in the browser. In addition, the CSS layout implementation is 
done by the authors. The authors have also actively participated in the W3C 
XForms Working Group. 
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This section describes the main architecture of the browser. The main com- 
ponents of the X-Smiles browser can be divided into four groups: XML pro- 
cessing, Browser Core Functionality, Markup Language Functional Components 
(MLFCs) and ECMAScript Interpreter, and Graphical User Interfaces (GUIs). 



3.1 Operation 

The core of the browser controls the overall operation of the browser. It includes 
browser configuration, event handling, XML Broker, etc. MLFCs handle different 
XML languages and render the documents. There are several GUIs in the X- 
Smiles distribution. They are used to adapt the browser to various devices or 
as virtual prototypes, when prototyping content targeted to diverse range of 
devices. [13] 

Basically, the X-Smiles operation consists of three steps: parsing, creating 
DOM, and rendering. Parsing and creating DOM are done concurrently. When 
reading new XML document, the X-Smiles browser first reads the document 
source. The file is accessed using either the file or http protocol. The Xerces 
XML parser and DOM implementation construct the DOM model of a document. 
Parser creates different Simple API for XML (SAX) events, which are used by 
DOM implementation. The MLFCs handle the presentation of the document. 
The MLFC layer is done in a modular way; it is possible to add and remove 
MLFCs for certain browser configurations without rebuilding the browser. It is 
also possible to show embedded documents. For instance, it is possible to use a 
SMIL presentation referenced by an object element in XHTML. 



3.2 Hybrid Documents 

Hybrid XML documents are documents, which contain several XML languages, 
separated by namespaces. Recently, the usage of hybrid documents has increased. 
The trend has been to specify XML languages as modules, which are combined 
to construct complete languages. This way, the modules can be reused across 
different languages, making implementations smaller and easier. It also reduces 
the number of different languages that the author must learn. 

As the number of languages gets higher, a way to flexibly handle these kinds 
of documents is required. The X-Smiles browser has a framework that handles 
hybrid documents. This framework includes a component called XML Broker, 
which handles the registration of language implementations. A language imple- 
mentation in the framework is called a MLFC (i.e., Markup Language Functional 
Component). An important concept in hybrid documents are host and parasite 
languages. There is always one host language in a document, usually determined 
by the namespace of the document element. The host language is used for the 
master layout of the document. Other languages that are embedded in the docu- 
ment are called parasite languages. In this paper, XHTML 2.0 is a host language, 
while XForms and XML Events are parasite languages. In an implementation, 
there is a need for communications between the host and parasite elements in 
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the DOM. In X-Smiles, this is done by extending the DOM implementation and 
implementing general browser-defined interfaces. [14] 

4 XHTML Module 

The XHTML module handles XHTML documents. The documents can be styled 
by style sheets and contain elements from other XML languages. The operation 
of the XHTML module is depicted in Fig. 3. An XML parser parses a given XML 
document and creates DOM document, which is an object tree representation of 
the XML document. From each tag in the XML document, a node for DOM tree 
is created. The document is displayed by a layout document, which implements 
Java Swing’s document model. The layout document is formed from DOM tree. 
Basically, every DOM element have respective layout element in layout docu- 
ment. Layout document is displayed by Swing Views. Each layout element has 
respective View, which defines its layout. Views render the document in browser 
window. [15] 




Fig. 3. Operation of the XHTML module 



The XHTML elements can be divided into two groups: stylable and non- 
stylable elements. All the visible elements are stylable. A style is assigned for 
each stylable element after DOM has been created. Elements get their style from 
the style sheet object. It is possible to change the visual type of the element by 
using the CSS property ’’display” (e.g., inline/block). 

Every DOM element has a respective layout element, which is used by layout 
document. In X-Smiles, XHTML elements and all the other elements in XHTML 
document have to have layout element in order to get rendered. In the case of 
XHTML, layout element is an instance of AbstractElement, which is an inner 
class of XHTMLDocument2. AbstractElement implements also Swing’s Element 
interface, whereas X HTMLDocument2 implements Swing’s StyledDocament in- 
terface. That way they both can be used as a part of Swing document model. 
[16] 

Views are also the part of the Swing document model. They define how 
elements are rendered in browser window. Views can contain content and other 
views and they are placed in a row either vertically or horizontally. For instance, 
inline elements are placed horizontally and paragraphs vertically. Every view has 
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some basic layout, which can be modified by style sheet. Elements are associated 
with Views by editor kit, which ties the layout document and views together. 

5 XForms Module 

The XForms module was also implemented using the MLFC interfaces in the X- 
Smiles browser. It is an parasite MLFC and always needs a host MLFC (in this 
paper, XHTML2 MLFC). The host is responsible for the main document layout, 
while the parasite is responsible for rendering the host language elements. 

5.1 Architecture 

Fig. 4 depicts the architecture of the XForms MLFC in X-Smiles. There are three 
main layers in the implementation: XForms model, Meta UI and User Interface. 
At the lowest level of the XForms model, there are the XML libraries that the 
implementation uses. For XML parsing and XML Schema, we used the Xerces-J 
2.4.0 implementation compiled with DOM level 3 support. DOM level 3 support 
was needed for Post Schema Validation Infoset (PS VI), which is used to get 
the data type information. For XPath, Xalan-J 2.5.1 was used. On top of the 
XML libraries lie the Instance document implementation, and the validation and 
calculation engine implementations. Data types are used in all layers, for instance 
to instantiate right types of form controls. The middle layer, Meta UI, contains 
high-level User Interface constructs, such as repeating items in a collection and 
switching parts of the User Interface on and off. The topmost layer is the User 
interface. To further enable multi-platform support, an additional layer, called 
Abstract Components was created. For each platform, an implementation, or 
wrappers, of the abstract components were included. Currently, we have support 
for AWT, Swing and Havi widgets. 



User 

interface 



Meta UI J 



XForms 
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Swing 1 1 AWT || Havi 


Abstract Components 




Form Controls | | Adaptive Controls | 



Switch 



Repeat 



Instance 
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Fig. 4. XForms MLFC architecture in X-Smiles 



XML Events was implemented by creating the XML Events MLFC into X- 
Smiles. The MLFC uses DOM events implementation in the Xerces-J parser to 
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implement listeners and events. A general action interface was created for the 
browser, and all XForms action elements implement that interface. 



5.2 CSS Integration 

Because of the way XForms language was designed, it was not entirely trivial to 
integrate it with a CSS based layout engine, such as the XHTML+CSS engine in 
X-Smiles. For instance, there exists a notion of cursor inside a repeated construct, 
that has no correspondence in the DOM. Also, the element describing the form 
control contains a label element inside it, making it difficult to style just the 
form control itself, as discussed in section 2.2. Basically, the implementation of 
pseudo-classes was quite straight-forward, since it is a concept outside of the 
DOM tree. On the other hand, pseudo-elements are conceptually in the DOM 
tree for CSS styling and cascade purposes, and are therefore much harder to 
implement. For instance, the pseudo-elements : :repeat-item and :: repeat-index 
should be inserted in the middle of the DOM tree. Fig. 5 depicts this; a simple 
repeat construct bound to a nodeset containing two nodes leads to more complex 
run-time object tree containing several pseudo-elements. Note that some DOM 
elements are treated as children of the pseudo-elements for purposes of CSS 
styling and cascading. 



crepeat nodeset="chapteisychapter"> 
<input ref="@title"> 




Fig. 5. Repeat-specific pseudo-elements 



We implemented all pseudo-classes, such as :invalid, describing the state of 
the form control. Also, we implemented the pseudo-element :: value. The pseudo- 
elements for styling the repeated structures were left as a future work item. 



6 A Case Study: Document Composition 

In this section, we introduce a real-world example of how to use XHTML 2.0 to 
compose and navigate a large document. One of the shortcomings of XHTML 1.0 
is that it does not support forms that edit repeating structures, thus requiring 
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server-side or script processing to provide such functionality. Another shortcom- 
ing is the non-existing support for navigation lists, such as pull-down menus. 
Navigation lists show only the currently selected part of the list, thus helping 
the user to navigate more easily in a large document hierarchy. Navigation lists 
have to be implemented with complex scripting in XHTML 1.0, decreasing acces- 
sibility and device-independence. The example shows how to use the declarative 
features of XHTML 2.0 to implement both the structural editing and navigation. 
The whole example runs in the client and does not need server-side program- 
ming. It also does not use any scripting, thus making it accessible and device 
independent. 

The document used in this case study is X-Smiles’ Technical Specification 
document. It has been created using the Doc Book XML format. In this case 
study, we show how to present and navigate that document in XHTML 2.0 and 
even edit the document composition. The Doc Book source files are controlled 
by XForms and transformed to XHTML 2.0 using XSL Transformations. 

The components of the example are shown in Fig. 6. The document consists 
of chapters, which each are in separate XML files. Headings, paragraphs, images, 
etc. are marked by specific tags in the chapter documents. All the chapter docu- 
ments are included into a composition document by XML Inclusions (Xinclude) 1 . 
The composition document is edited by Edit document, which is an XHTML 2.0 
document with XForms form. The composition document is an instance of the 
form. Through the Edit document, user can select, which chapters are included 
in the final document. The final document is transformed to XHTML 2.0 format 
by XSL style sheet. The document composition and structure are discussed in 
more detail below. 




Fig. 6. Test case architecture 



The editor document, which is depicted in upper-right corner of Fig. 6, is 
an XHTML 2.0 document, which includes embedded XForms elements. The 
XForms instance data is the Composition document. The editor document uses 
a XForms repeat element to create an editor for this document. It is possible to 
add new sections to the composition, or enable/disable them. The editor UI is 
depicted in Fig. 7 (a). 

1 XML Inclusions, http://www.w3.org/TR/xinclude/ 
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(a) 
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Fig. 7. (a) Editing the composition with XForms and (b) viewing the final document 



The XSL style sheet converts all the elements from the composed file to 
XHTML 2.0 elements. Content of chapter documents is divided to sections, 
which can contain other sections, paragraphs, headings, etc. The XSL Transfor- 
mation converts all the chapter and section elements to XHTML 2.0 sections. 
Therefore, document is structured automatically. In XHTML 2.0, all headings 
can be represented by h elements. Their style depends of amount of embedding 
sections. So, during transforming, there is no need to decide the level of headings. 

The document has a menu in the beginning. It is realized using XHMTL 2.0 
navigation list. Navigation list contains all the headings of the document. The 
navigation list shows only selected part of the list at the time. List items are 
also links, which refer to the heading in question in the document. Source code 
of the navigation list with few entries is shown below. 

Source code of the navigation list 

<nl> 

<label>Table of Contents</label> 

<li href ="#Introduction"> 

<nl> 

clabel href ="#Introduct ion" >Introduction</label> 

<li href ="#WhatisX-Smiles?">What is X-Smiles?</li> 

<li href ="#MainFeatures">Main Features</li> 

<li href ="#ThisDocument">This Document </li> 

</nl> 

</li> 

<li href="#Environment"> 

<nl> 

clabel href ="#Environment " Environment </label> 
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<li href="#RuntimeEnvironment">Runtime Environment</li> 

<li href="#Runningit">Rimning it</li> 

<li href="#BuildingwithAnt">Building with Ant</li> 

</nl> 

</li> 

</nl> 

The navigation menu could be styled like a regular pull-down menu as shown 
in Fig. 1, but due to limitations in the CSS layout in the current version of X- 
Smiles (0.82) that was not done. As discussed in Section 8, we are building a 
better CSS layout engine into the X-Smiles browser. The resulting document, 
viewed in the X-Smiles browser is shown in Fig. 7 (b). Part of the navigation 
list, whose source is shown above, is also visible. 

7 Discussion 

So far, the appearance of a new HTML version has not caused much headache 
for authors and user agent developers. Different HTML versions are backward 
compatible, which means that old documents do not need major revising. Even 
the transition to XHTML 1.0 is a straight forward process as it is basically a 
reformulation of HTML 4.01 in XML format. The new XHTML 2.0 specification 
will cause more problems, though. The biggest change is related to forms. The 
fifth Working Draft specifies that XForms will be used as the forms technology 
in XHTML 2.0. 

XForms requires major changes especially in user agents. Old HTML form 
implementations cannot be used as a base for XForms, because the whole concept 
is different. XForms has a new data model and it contains more functionality 
than HTML forms. The new data model requires the use of XML Schema and 
XPath processors. Both of these are large components, which causes problems 
in restricted devices. W3C is aware of this problem, and thus it has defined the 
XForms Basic Candidate Recommendation. In XForms Basic, the data model is 
less advanced, requiring only support for certain data types, in effect removing 
the need for full XML Schema processor. 

The transition to XHTML 2.0 causes also problems for authors. The main 
obstacle is that they have to learn a new forms language. Fortunately, they do not 
have to start from scratch. The reason is that XForms is based on XPath. Most 
XML developers use XSLT regularly. Since both XForms and XSLT are based 
on XPath, the developers already have a starting point. Another advantage is 
that XForms is declarative language. Current, HTML forms are heavily based on 
scripting, which makes updating and reusing of forms difficult. XForms is easier 
to maintain and it allows the reuse of forms in different XHTML documents. 

Another major change in XHTML 2.0 is the real separation of content and 
presentation. XHTML 2.0 removes all the presentation related features from 
XHTML 1.0 and concentrates only on defining the structure of the document. 
All presentation related issues are defined using separate CSS style sheets. In ad- 
dition, several XHTML 2.0 elements and attributes contain functionality, which 
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before had to be realized using scripting. Therefore, authors do not have to 
use scripting as much as before and also their documents will be more widely 
accepted by the different user agents. Both of these features improve the main- 
tenance of XHTML 2.0 documents. 

Based on the above facts, our conclusion is that XHTML 2.0 is a major step 
in the development of web technology. The new features require major changes 
in user agents, though. In addition, the authors have to learn how to use the 
new features. In our opinion, the advantages brought by XHTML 2.0 outweigh 
the problems. For instance, XHTML 2.0 removes the need for heavy scripting 
and separates the content from presentation. Therefore, XHTML 2.0 documents 
are easier to maintain and reuse. 

8 Conclusions 

In this paper, we have discussed the new XHTML 2.0 specification and its user 
agent implementation. According to the previous section, the XHTML 2.0 doc- 
uments are easier to maintain and reuse, because they require less scripting 
and the content is really separated from the presentation. The introduction of 
XHTML 2.0 requires major changes in user agents, though. 

Our XHTML 2.0 implementation is part of the X-Smiles user agent. The 
implementation is modular and mainly based on already existing components. 
No changes had to be made to the parsing and processing of XML documents. 
The XHTML module uses the well established DOM interface to access the XML 
data. The DOM documents are rendered using Java Swing document model. The 
XHTML 2.0 documents can be styled by CSS style sheet and they can contain 
XForms elements. We reused an already existing CSS processor. 

Unfortunately, we could not implement all XForms specific CSS features. 
Currently, we are removing the Swing dependency form the CSS layout engine. 
In future, we plan to use the new CSS layout engine with the XHTML 2.0 and 
XForms components. Then, we expect to be able to implement all XForms spe- 
cific features. For instance, at the time of writing we have already implemented 
all the XForms pseudo-elements in the new version of the layout engine. 

The results of this paper show that implementation of XHTML 2.0 requires 
further development of user agents, but is not unrealistic. User agents, which 
already support XHTML 1.0 can be updated to support XHTML 2.0. Already 
existing components can be used in most cases. Generally, changes below the 
DOM interface are not required, albeit XForms requires DOM Level 3 support. 
Also, existing XML Schema and XPath engines can be used. Some changes to the 
CSS layout model can be expected, though. In some cases, the XForms module 
is the only new component. 

In our future work, we plan to study the use of XHTML 2.0 in restricted de- 
vices. The main research question is how all the components required by XHTML 
2.0 can be fitted in to a restricted devices with less memory and processing power 
than desktop devices. Most problems are related to XForms. At the moment, it 
remains to be seen whether XForms Basic will help in this problem. 
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Abstract. In this paper, we propose a method to automatically rank 
documents returned by a search engine in the WWW with respect to a 
query. The process consists in three steps, the first translates the query 
and document descriptions into description logic terminologies. The 
second computes a mapping between related elements in the query and 
each document. This mapping matches concepts in the terminologies 
based on their names and their definitions. The last step computes 
the difference between the query (represented as a terminology) and 
each document (also represented as a terminology) and ranks the 
documents according this difference. To deal with linguistic information 
when comparing description logic concepts, we propose a definition of 
subsumption that takes into account names similarity between concepts 
occurring in the descriptions being compared. We describe each step of 
the method and show the intended results on a running example. 



1 Introduction 

This paper deals with the problem of ranking documents returned by a search 
engine in the WWW. The ranking function involves a semantic comparison be- 
tween the client query and the documents content. The document that best 
matches the query is returned first. To perform this ranking we need: 

— A formal description of the query and the documents content: the query and 
the documents are specified in natural language (NL), a formal description 
of their semantics is needed to achieve an automatic comparison. 

— A matching algorithm that compares a query description and a document 
description and returns a set of matching elements between the query and 
the document. 

— A ranking function that sorts the documents with respect to the size of the 
part of the query that is not covered by the documents. 
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The Proposed Approach 

We propose to use description logics (DLs) [2] as a formal representation lan- 
guage for specifying documents and queries. The motivations are twofold: DLs 
come with well-defined semantics and correct inference algorithms and the for- 
malization of a text in DLs has already been studied, see [8] for details about 
how we extract a terminology from a natural language document. As already 
stated in [14], not all aspects of a natural language can be captured by a formal 
description, we will restrict ourselves to a small fragment of NL that can be 
represented in DLs. 

The matching step consists in comparing the two terminologies obtained from 
a query and a document. Given two terminologies Tq and T d describing a query 
Q and a document D respectively, our goal is to find the elements in T q and T o 
that match. This is done by a matching function that takes two terminologies as 
input and produces a one to one mapping between defined concepts of the two 
terminologies that correspond semantically to each other. 

Finally, the documents are ranked according to the size of the extra infor- 
mation contained in the query and not in the documents. The extra information 
is calculated with the help of the difference operation between pairs of mapped 
elements. We propose an algorithm that computes the difference between AC£- 
concept descriptions. It is based on the work reported in [10], by taking into 
account linguistic relations (synonymy, hypernymy...) between concept names 
occurring in the two descriptions. 

The paper is organized as follows. Section 2 presents the motivation behind 
this work. Section 3 gives a brief overview of description logics and the difference 
operator. We define the matching and the ranking problems in sections 4 and 5 
respectively. We conclude in section 6. 

2 Motivation 

All major search engines available on the web rank web pages by determin- 
ing relevancy through analyzing keyword location, frequency and through other 
methods, for example, by analyzing how pages link to each other. Non relevant 
pages appear frequently in the resulted rank and it may take time for the user 
finding out the intended information among this huge number of pages. 

Our goal is to allow the user to describe his requirements by specifying a 
detailed description so that we can compare it semantically to the content of 
web pages. The most relevant page compared to the query needs comes first. 

The proposed algorithm detects the parts of the documents that are seman- 
tically related to some parts of the query and then deduces the non covered part 
in the query. The most relevant match will be the one with the smallest non 
covered part. 

Let us illustrate the practical interest of our method with an example. 

Example 1. Consider the three simple NL texts depicted in Figure 1. The query 
Q describes the rooms of a hotel, Di and D 2 are documents returned by a 
research engine. 
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Q 


All the rooms are comfortable and air conditioned. Each room is provided with a 
TV and a large bed. The hotel is located in Paris. 


D 1 


Located in the french capital, the hotel is convivial. Each bedroom is air conditioned 
and provided with a cable television. The breakfast is served in an elegant lounge. 


d 2 


The hotel is located in Paris. All our homelike rooms are provided with a queen 
size bed, a color TV and a private bathroom. The hotel accepts only credit card 
guarantee. 



Fig. 1 . Simple NL texts 



By reading the documents, a user can see that the second document matches 
better the query needs than the first one. To do this, we need to be able to de- 
tect the related parts between the query and each document. The first document 
shares with the query the same information about hotel location, air condition- 
ing and the existence of a television. For the second document, the common 
information concerns hotel location, comfort of the rooms and the existence of 
a television and a large bed. 

This discovery is challenging because different words can be used to express 
the same semantic information. This can be done by synonyms (e.g. comfortable 
vs homelike , french capital vs Paris ) or hypernyms (e.g. bedroom vs room, cable 
television vs TV). 

Once this matching performed, the extra information contained in the query 
and not in the document can be computed easily. It is clear that the document 
that better meets the user needs is the one with the minimal extra information. 



3 Preliminaries 

In this section, we introduce description logics, the formalism used in our frame- 
work, the difference operator and the notion of size of a description. 



3.1 Description Logics 

Description logics (DLs, also called terminological logics) are a family of knowl- 
edge representation formalisms designed for representing and reasoning about 
terminological knowledge. In DLs, the conceptual knowledge of an application 
domain is represented in terms of concepts (unary predicates) that are inter- 
preted as sets of individuals, and roles (binary predicates) that are interpreted 
as binary relations between individuals. 

Starting with the set Nc of concept names and the set Nr of role names, 
complex concept descriptions are built inductively using concept constructors. 
The different description logic languages distinguish themselves by the kind of 
constructs they allow. In our framework we are going to use the ACE description 
logic. In this description logic, concept descriptions are formed according to the 
syntax rules depicted Figure 2. A £ Nq denotes a concept name, r £ Nr a role 
name, and C, D (complex) concept descriptions. 
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C, D ->• T I 


(top-concept) 


T| 


(bottom-concept) 


A | 


(concept name) 


-4 1 


(primitive negation) 


CnD 


(conjunction) 


3r.C | 


(existential restriction) 


Vr.C | 


(value restriction) 



Fig. 2. Syntax of some concept descriptions 

Let C denotes some description logic, a concept built using the constructors 
of C is called an ^-concept. 

The semantics of a concept description is defined by the notion of interpre- 
tation as given below. 

Definition 1. (Interpretation) An interpretation X = (A x ,. x ) consists of a 
non-empty set A 1 , the domain of the interpretation, and an interpretation func- 
tion x that maps each concept name A £ Nq to a subset of A 1 and each role 
name r £ Nr to a binary relation r x , subset of A x x A 1 . The interpretation 
function can be extended to arbitrary concept descriptions as shown in Figure 3. 



T 1 = A 1 

l 1 = 0 

(~<A) X = A 1 \ A 1 
(c n D) 1 = c x nD x 

(3 r.C) x = {x £ A x | 3 y : (x, y) £ r x A y £ C 1 } 
(\/r.C) x = {a; £ A x j Vy : (x, y) £ r 1 — > y £ C 1 } 



Fig. 3. Semantics of concept descriptions 



DL systems provide various reasoning services, the most important is the 
computation of the subsumption relation. 

Definition 2. (Subsumption) Let C, D be concept names, D subsumes C (noted 
CQD) iffC 1 C D 1 for all interpretation X. 

Concept descriptions are used to specify terminologies that define the intentional 
knowledge of an application domain. 

Definition 3. (Terminology) Let A be a concept name and C a concept defini- 
tion. Then A = C and A AC are terminological axioms. The first is a complete 
definition, the second an incomplete one. A terminology T is a finite set of ter- 
minological axioms such that no concept name appears more than once in the 
left-hand side of a definition. If a concept A occurs in the left-hand side of a 
definition, it is called defined concept. The other concepts are called primitive 
concepts. 
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A terminology built using the constructors of some description logic C is called 
an T-terminology. 

An interpretation I is a model of a terminology T if it satisfies all the 
statements contained in T : 

- A 1 = C 1 for all terminological axioms A = C in T, 

- A 1 C C x for all terminological axioms A C C in T. 

In our work, natural language documents are represented by a DL terminol- 
ogy, NL statements are transformed into terminological axioms. 

Example 2. The NL texts of example 1 are represented by the ACS - 
terminologies Tq, Td-i and T d 2 given in Figure 4. 



Tq 


Room = Comfortable n Air-conditioned n dprovided-with.TV n 
3provided- with. (Bed n 3has-size. Large) n Rooulq 
Hotel = 31ocated-in. Paris n HotelQ 


t Di 


Hotel = 31ocated-in. French-capital n Convivial n Hoteluj 
Bedroom = Air-conditioned n 3provided-with. Cable-television n Bedroom_Di 
Breakfast = 3served- in. (Elegant n Lounge) n Breakfastu! 


Td 2 


Hotel = 31ocated-in. Paris n Vaccept. Credit-card-guarantee n Hotel_o 2 
Room = Homelike n 3provided- with. (Bed n 3has-size. Queen-size) n 
3provided- with. Color-TVn3provided- with. (Bathroomfl Private) n 
Room£> 2 



Fig. 4. Examples of terminologies 

The overlined concepts stand for the missing part of the definitions, we will 
ignore these concepts in the rest of the paper. 

3.2 The Difference Operator 

Informally speaking, the difference between two concept descriptions is the in- 
formation contained in the first description and not in the second. The difference 
operator allows to remove from a given description all the information contained 
in another description. The difference operation between two concept descrip- 
tions was first introduced by Teege [15]. The difference between two concept 
descriptions C and D with CCD is given by 

C — D := max{E \ E n D = C} 

where max is defined with respect to subsumption. 

[10] proposed a refinement of this definition by allowing the difference be- 
tween incomparable descriptions (i.e. D is not required to subsume C) and taking 
the syntactic minimum (w.r.t a subdescription ordering A<j) instead of a seman- 
tic maximum. The difference between two incomparable concept descriptions C 
and D is defined as 



C — D := min{E \ E n D = C n D} 
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where min is defined with respect to a subdescription ordering. 

This definition has two advantages, it does not contain redundancies and it 
is more readable by a human user. However, Tegee’s difference captures the real 
semantic difference between two concept descriptions. 

We use the second definition because it is defined for the DL ACE that allows 
us to represent a considerable number of NL semantics. 



3.3 Size of a Concept Description 

We define the size \C\ of an _4££-concept description C as the number of con- 
juncts occurring on the top-level of C . 

Example 3. The sizes of the concepts Room and Hotel of ffi q in Example 2 are 
5 and 2 respectively. 



4 The Matching Algorithm 

In this section we introduce the matching operation. Match is an operation that 
takes a query description and a document description and returns a mapping 
that identifies corresponding elements in the two descriptions. This mapping 
consists of a set of mapping elements indicating that certain elements of the 
query Q are related to certain elements of the document D. By elements, we 
mean the defined concepts in the terminologies T, q and ffi d- A concept A: from 
Tq is related to a concept Bi from Td if their names and their definitions are 
similar. 

In [11], a schema matching algorithm called Cupid was proposed. A schema 
consists of a set of related elements such as database or XML elements. The result 
of the match operation is a mapping indicating that certain elements of the first 
schema are related to certain elements of the second. Similarity coefficients are 
computed between elements of two schemas in two phases, a linguistic and a 
structural one. Then a mapping is deduced from those coefficients. 

Following Cupid intuitions, we build an algorithm for matching a query de- 
scription and a document description. First, it proceeds by computing similarity 
coefficients between defined concepts in the two terminologies and then deduces a 
mapping from those coefficients. The coefficients in the range [0,1] are calculated 
in two steps: 

— Step 1 . Matching of names : it is based on the notion of semantic relatedness 
introduced in [7] that measures the extent to which two lexicalized concepts 
are close. This measure is based on the semantic relations of Wordnet [6]. 
We will call the result the name similarity coefficient ( nsim ). 

— Step 2. Matching of description, it consists in comparing the concept de- 
scriptions occurring in the two terminologies. This phase uses name similar- 
ities between concepts appearing in the concept descriptions. We will call 
the result the description similarity coefficient ( dsim ). 
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The weighted similarity ( wsim. ) is a mean of nsiin and dsim, it is calculated as 
follows: wsim. = w x nsim + (1 — w) x dsim , where w is a constant in the range 
[0,1]. We compute weighted similarity coefficients between defined concepts in 
the terminologies. A mapping p is deduced from those coefficients by choosing 
pairs of elements with maximal weighted similarity. 

In the next two subsections, we detail name matching and description match- 
ing steps. 



4.1 Name Matching 

The first step of the matching is based on defined concept names. We need 
to determine the degree of semantic similarity between two concept names. We 
reuse the notion of semantic relatedness between two lexically expressed concepts 
[7]. This measure uses WordNet [6] as knowledge source. The idea behind this 
measure is that two concepts are close if the path relating them in WordNet 
is not long and does not change direction too often. We recall the definition of 
semantic relatedness and define the name similarity coefficient. It is expressed as 
function of the semantic relatedness since we require it to be in the range [0,1]. 

Definition 4. (Semantic relatedness) [7] The semantic relatedness of two con- 
cept names c\ and C 2 is given by: 

rel(ci, C 2 ) = C — PathLength(c\,c- 2 ) — k * NumberOfChangesOfDirection(ci, C 2 ), 



where C and k are constants and PathLength denotes the length of the shortest, 
path, between two concepts. If no such path exists, rel(c\,C 2 ) is zero. 



Definition 5. (Name similarity coefficient) The name similarity of two concept 
names Pi, P 2 & Nc is given by: 



nsim(P\, P 2 ) 



rel{Pi,P 2 ) 

C 



where C is the same constant used in the definition of the semantic relatedness. 



4.2 Description Matching 

The intuition behind the description matching is that two concept descriptions 
are similar if their difference is minimal. We estimate the description similarity 
coefficient as a function of the size of the difference between the two descriptions. 



Definition 6. (Description similarity coefficient) The description similarity be- 
tween two concept descriptions C and D is given by: 

\C-D\ 

\C\ 



dsim(C, D) = 1 
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The algorithm proposed in [10] that performs the difference between two 
AC£-concept descriptions is based on the subsumption test. We propose a new 
definition of the subsumption, that takes into account linguistic information 
about concepts occurring in the descriptions being compared. It is denoted by 
Cg. In order to exploit the graph-based subsumption reasoning, we are going to 
represent the concepts occurring in the terminologies as trees. 

We know that a tree-based characterization of subsumption was stated in 
[1]. It works in three steps. First, concept descriptions are turned into normal 
forms, that makes the knowledge implicitly contained in a concept description 
explicit. Second, these normal forms are translated into description trees. Then 
subsumption is characterized in terms of graph homomorphism between the 
description trees. 

We first recall the definition of normal forms and description trees, then we 
propose a definition of homomorphism taking into account name equivalence 
between concept names occurring in the descriptions. 

Definition 7. (ACS -normalization rules) Let C, D be two ACS-concept de- 
scriptions and r £ Nr a primitive role. The ACS -normalization rules are defined 
as follows 



\/r.C n Vr.-D — > Mr.(C n D) 

Mr.C n 3 r.D -> Mr.C 13 3 r.(C 13 D) 
Mr. T T 
<313 T -M3 

P 13 ->P — > _L,/or each P £ Nc 
3r.J_ — ► T 
C 131 M 1 



If only the rule Mr. T — > T is applied to a concept description <3, the resulting 
concept is called in T -normal form and the corresponding description tree is 
noted Qq. 

Definition 8. (ACS -description trees) An ACS -description tree is a tree of the 
form Q = (N, E, no, £) where 

— N is a finite set of nodes ofQ; 

— E C N x (Nr U MNr) x N is a finite set of edges labeled with role names r 
(3-edges) or with Mr (M -edges); MNr := {Mr \ r £ Nr}; 

— no is the root of Q; 

— £ is a labeling function mapping the nodes in N to finite sets {P\, ...,Pk} 
where each Pi, 1 < i < k, is one of the following forms: Pi € Nc, Pi = _l T > 
for some P € Nc, or Pi = _L. The empty label corresponds to the top-concept. 

For n,m £ N and r £ Nr, an 3-edge from n to m labeled r is written as nrm, 
and a M-edge as nMrm. 
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Every ACE -concept description C can be turned into an ,4££-description 
tree Gc (see [1] for a formal definition of this translation) . 

Example 4- The ACE -concept description 

Room = Comfortable n Air-conditioned n dprovided-witlr.TV n 
dprovided- with. (Bed n 31ras-size. Large) 

yields the tree G Room of Figure 5. 



Definition 9. (Name equivalence) Let Pi and P 2 be two concept names in Nc, 
Pi and P 2 are said to be equivalent, written Pi = P 2 , if nsim(Pi, P 2 ) exceeds a 
certain threshold th S i m . 

Given a set S of name equivalences between concept names and two ACE- 
description trees, we define the notion of homomorphism between description 
trees w.r.t a name equivalence set as follows. 

Definition 10. (Homomorphism on description trees w.r.t a name equiva- 
lence set) A mapping <p : Nh —> Nq from an ACE -description tree H = 
( NH,EH,rno,£H ) to an ACE -description tree G — (Nq, Eg, no, Ig) is called 
homomorphism w.r.t a name equivalence set S, if and only if the following con- 
ditions are satisfied: 

1. <p(m 0 ) = n 0 ; 

2. for all n €E Njj we have, VPi £ £jj(n),3Pj £ ^G( < / ? ( n )) such that Pi = Pj is 
in S or - L £ tcitpinf); 

3. for all nrm £ Eh, either <p(n)rip(m) £ Eq, or ip(ri) = <^( TO ) an d -L € 
^g(v ? («))/ and 

4- for all vNrm £ Ejj, either ip(n)\/np(m) £ Eg , or 1 p(n ) = <p(m) and J_ £ 
e G (c{n)). 
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Fig. 6. Subsumption between _4££-description trees 



Theorem 1 . Let C,D be ACE-concept descriptions. Then, C C5 D iff there 
exists a homomorphism w.r.t a name equivalence set S from Qj, to Qc- 

Sketch of proof. The proof is based on the one given for the theorem 41 in [1], 
The idea is to use at different stages of the proof the fact that if P = P' we have 
£0 € P 1 implies that Xq € P ' 1 and vice versa. 

Example 5 . Let us illustrate Theorem 1 by two concept descriptions, namely 
Room and Bedroom 

Room = Comfortable n Air-conditioned n 3provided-with.TV n 
3provided- with. (Bed n 3has-size. Large) 

Bedroom = Homelike n ^provided- with. Color TV n 3provided- with. (Bed n 
3has-size. Queen size) 

and the name equivalence set S 

S = {Comfortable = Homelike, 

TV = Color TV, 

Large = Queen size} 

The descriptions are already in a normal form. A homomorphism from Q Bedroom. 
to Q Room w.r.t S is depicted in Figure 6. From Theorem 1 we can conclude that 
Room C5 Bedroom. 

vvvvvvvv 

Based on the algorithm proposed in [10] for computing the difference between 
A££-concept descriptions, we propose the algorithm diff S j m depicted in Figure 7. 
Two changes have been made to the original algorithm. First, the definition 
of prim(C-D) has been changed since we are dealing with a name equivalence 
operator instead of equality of concept names. Second, the proposed subsumption 
over a set of name equivalence is used instead of the classical subsumption (line 
6). The following notations are used: 
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— prim(C) denotes the set of (negated) concept names and the bottom concept 
occurring on the top-level of C, 

— C.r = E if there exists a value restriction Vr.E on the top-level of C\ C.r — T 
otherwise, 

— 3 r.C' £ C means that 3r.C" occurs on the top-level of C . 



Require: AC£-concept descriptions C and D in ALE- normal form, a set S of name 
equivalences. 

Ensure: diS s im(C, D) 

1: if C n D = _L then 
2: diff s ; m (C, D) := _L 

3: else 

4: diff sim (C, D ) := n Ae P rim(C-D)A n Vr. diff sim (C.r, D.r) n n B6£ v 3 r.E 

where prim(C — D) := {P £ prim(C) \ there does not exist P' £ prim(D) with 
P = P' £ S} and the value restriction is omitted in case diff s i m (C.r, D.r) = _L 
and E' r is computed as follows: 

Let 3r.Ci, ..., 3r.C„ £ C, 3r.Di, ..., 3r.D m £ D be all the existential restrictions 
in the top level of C and D, respectively, £ r = (Ci, ..., C n }. 

5: for i = 1 to n do 

6 : if (i) there exists j £ (1, ^ i,with D.r 13 C.r n Cj Cs Ci, or 

(ii) there exists j £ { 1, ..., m},with D.r n C.r n Dj C 5 Ci, then 
7: E r ~£r\{Ci} 

8: end if 

9: end for 

10: £' r = {E* | E £ £ r } where E* := diff a im{E, C.r n D.r). 

11: end if 

Fig. 7. The algorithm diff s i m 



Example 6. Computing the difference between the concept descriptions of Ex- 
ample 5 yields 



Room — Bedroom = Air-conditioned 



4.3 Mapping Generation 

The set of mapping elements is deduced from the computed name and description 
similarities. Let Tq = {Qi = Ci,i £ [l,n]} and Td = {Dj = Cj,j £ [l,m]} be 
two A££-terminologies describing a query Q and a document D respectively. A 
mapping p from Tq to Td is computed as follows: 

— p(Qi) = Dj, 1 < i < n, 1 < j < m, if wsim(Qi, Dj) > th mav and 
wsim(Qi, Dj) > wsim{Qi , Dk) for all D k £ T d, k ^ j , 

— p(Qi) = T, if there is no Dj, 1 < j < to, with wsim(Qi , Dj) > th m ap- 




Semantic Matching of Natural Language Web Queries 427 



Example 1. Let us illustrate the matching step on the terminologies Tq and 
T d-i of Example 2. The set of computed mappings is depicted in Table 1. The 
name similarity coefficients nsiin are computed with (7 = 8 and k = 1. The 
chosen thresholds th S i m and th ma p are 0.75 and 0.5 respectively. For wsim, we 
take w = 0.5. 



Table 1 . The mapping generated from Tq and T d x 



p 


nsim 


dsim 


wsim 


Room — » Bedroom 


0.87 


0.5 


0.68 


Hotel — * Hotel 


1 


1 


1 



5 The Ranking Problem 

In this section we show how the mapping generated by the matching algorithm 
is used to compute the difference between a query terminology and a document 
terminology. Then we show how documents are ranked with respect to this dif- 
ference. 

Let Tq = {Qi = Ci,i € [l,n]} and Td = {Dj = Cj,j £ [l,m]} be two 
7l££-terminologies describing a query Q and a document D respectively. The 
difference between the two terminologies is defined as follows: 

Definition 11. (Difference between terminologies) Given the mapping p result- 
ing from the matching between Tq and Td- The difference between the termi- 
nologies T q and T d is the conjunction of the differences between each related 
pair of concepts in the two terminologies. 

dif f p {Tq,T d) = ^Qi£T Q {Qi ~ p{Qi )) 



With the notion of size of a description, we define the dissimilarity coefficient 
between two terminologies. 

Definition 12. (Dissimilarity coefficient) The dissimilarity coefficient between 
the terminologies T q and T d is the size of their difference 

d(T Q ,T D ) = \diff p (T Q ,T D ) | 



Documents are ranked with respect to the dissimilarity coefficients between 
their descriptions and the query description. The more a document covers the 
query, the best is its rank. 
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Example 8. Let us now illustrate the ranking process on the terminologies Tq, 
T d x and T d 2 of Example 2. The differences between the terminologies are the 
following 

dif f P {TQ,T dJ = Comfortable n dprovided- with. (Bed n 3has-size. Large) 
d*//p(Tq,T d 2 ) = Air conditioned 

We can deduce that the second document matches better the query than the 
first since its difference is the smallest one. Hence, we have the intended result 
described in Section 2. 



6 Discussion 

Nowadays, search engines sort their results according to number of criteria going 
from the number, proximity and location of terms matched, to pages related 
factors such as the number of links made to a page or the number of times a 
page is accessed from a results list. The ranking algorithms used by the search 
engines are not published and we know only a little about their ranking criteria 
[9]. The novelty of the approach proposed in this paper is that it allows the 
user to express his query as a natural language description. The criteria used 
when sorting the retrieved documents is their semantic relevancy w.r.t the query 
needs. 

In the semantic web framework, an approach of ranking query results is 
proposed in the SEAL semantic portal [13]. Query results are reinterpreted as 
F-Logic knowledge bases. The semantic ranking is reduced to the comparison 
of two knowledge bases. A similarity is computed between the query and the 
knowledge bases, it serves as a basis for the semantic ranking. Their notion of 
similarity between two terminologies is reduced to the similarity between concept 
pairs. 

Number of similarity measures for ontological structures were proposed in 
different domains like databases, artificial intelligence and semantic web [12,3, 
5]. The work in [12] extends the comparison to semantic structures (set of super 
and sub-concepts of a concept) and relations between the concepts. 

Our approach in matching terminologies is more complete since it operates 
in semantic descriptions expressed in description logics rather than structures. 
In addition, it involves both name and complex description matching. 

Our name similarity measure is based on Wordnet. Different measures be- 
tween lexicalized concepts in the Wordnet hierarchy have been proposed (see [4] 
for a survey). We choose to use the measure proposed by Hirst and St-Onge [7] 
because it uses all relations in Wordnet, the other measures are based only on 
hyponymy. 

The constant and threshold values proposed in example 7 for computing the 
mapping between two terminologies are the typical values we have used in our 
experiments. Those values are subjective and depend on the intended results. For 
example, for additional restriction of the the mapping, th map can be increased 
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and allow only for synonyms and direct hypernyms in the set of equivalence 
names, th S im have to be set to 0.87. 

In real-life applications, often exact matching is not realistic. We will inves- 
tigate in our future work an approximate matching of the query results. 
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Abstract. A vast amount of information resources is stored as 
relational-like data and inaccessible to RDFS-based systems. We de- 
scribe FDR2 - an approach to integration of relational-like information 
resources with RDFS-aware systems. The proposed solution is purely 
RDFS-based. We use RDF/S as a mechanism to specify and perform 
linking of relational data to a predefined domain ontology. The approach 
is transformation-free, this ensures that all the data is accessible and 
usable in consistence with the original data model. 



1 Introduction 

The RDF and RDFS languages have been developed to express machine under- 
standable semantics to facilitate more intelligent ways of information processing. 
RDF/S languages provide a unified syntax, data model, well-defined semantics 
and enable separation of data (RDF) from meta-data (RDFS). The formal ac- 
ceptance of RDF/S by W3C [1] stimulates their utilization in many areas and 
by many organizations. 

In spite of an increasing acceptance of RDF/S, this is still a new technology. 
Most information resources are not available in RDF/S-format. The relational 
data model, on the other hand, is widely accepted and currently supported by 
thousands of applications ranging from simple spreadsheets to complex relational 
databases. 

Within this paper we use relational data model to refer to data presented 
as a collection of records usually depicted as a table. We would like to note 
that within the paper we differ between relational data model and relational 
database model. The latter extends the former by assuming that columns repre- 
sent attributes, rows represent entities, and the table contains a set of attributes 
uniquely identifying each entity. We did not accept such assumptions because 
our experience indicated that very often the users organize data in an ’’intu- 
itive” tabular way supported by spreadsheet applications but incompatible with 
these database-specific assumptions. This made us to focus on the less restricted 
relational data model. 

Our application interest is to bring RDF/S technology to R&D companies 
and institutes as a part of what has been labelled as e-Science. The idea is 
that results of scientific experiments and computations as such should be shared 
within the (global) scientific community, in addition to communicating through 
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traditional scientific publications. The latter do not contain sufficient details 
of the work done and the information presented by them cannot be machine 
processed. 

Sharing is to be supported by an ontology-based information system. The 
primary goal of the system is to assist with the transition from traditional ex- 
perimental science to e-Science facilitating large scale collaboration between sci- 
entists. Since we intend to bring benefits of ontologies into the e-Science envi- 
ronment, we have to find a way to link the relational data to an RDF/S model. 

The main objective is to allow ontology-based querying of the relational data: 
the original relational data must be made available to an RDFS reasoner and 
become queryable with a vocabulary predefined in a domain ontology. 

Our approach to the linking problem is explained in Sect. 2. Section 3 dis- 
cusses the presented method and related research and Sect. 4 summarizes the 
results of this work. 

2 FDR2 Approach 

We present our approach as a general procedure that can be modified and ex- 
tended to fit particular needs. Let us assume that we have a set of data, expressed 
in a relational way (e.g., as a spreadsheet table where the first row is a header) 
and a domain ontology (DO) expressed in RDFS. 

The general problem of linking relational and RDF /S models can be broken 
down into two subproblems: 

1. Expressing relational data in an RDF/S format to enable syntactic and struc- 
tural interoperability with DO. 

2. Linking the newly created serialization to the RDFS representation of DO. 

We approached the first subproblem by performing a two-step serialization of 
the relational data. On Step 1 we automatically create a relational schema (RS) 
to express and preserve structure of the original data ( data representation level 
on Fig. 1) and to provide a foundation for interoperability with DO ( pre-RDFS 
level on Fig. 1). The data representation level contains notions common for all 
relational data sources (a header, rows, columns and cells). Concepts defined on 
the pre-RDFS level are shared between data sources with the same structure 
(table header) . On Step 2 we use RDF to express the actual content of the table 
according to the RS. 

The second subproblem cannot be solved automatically due to the undefined 
semantics of the relational data. We leave it to the user to define the relationships 
between RS and DO (Step 3). 

Our approach consists of three major steps. Additional actions (minor) are 
needed to exploit the RDF/S documents created during those steps to enable 
run-time interoperability (ontology-based querying). 

Step 1: Build a relational schema to explicitly define the underlying relation. 
Since we cannot assume that every column represents an attribute and every 
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row represents an entity, we have decided to build relational schemata upon a 
notion of class. 

Every header cell represents a name of 
a class that is extensionally defined by the 
column cell values. An ordered set of all 
such classes C defines the underlying re- 
lationship R expressed within the table. 
RDFS models explicitly support binary re- 
lations only. Since any binary relation de- 
fined over a set A is a subset of Ax A, 
where x denotes a Cartesian product, we 
have decided to use CxC to obtain all bi- 
nary virtual relations ( virtual properties) 
defined over the classes presented in the 
relational schema. 

After this step we have obtained the 
relational schema expressed in RDFS and 
Fig. 1. The structure of the relational containing a definition of all classes and 
schema. all binary relationships between them. The 

resulting RS serves as a compact (inten- 
sional) representation of the data and can be directly connected to DO. 

The structure of the relational schema is depicted on Fig. 1. Dashed arrows 
indicate transformation of simple concepts into more complex ones. For example, 
a collection of cells becomes a column which at the end is generalized into a class. 
Solid arrows explicitly indicate that classes and virtual properties enable access 
to the actual table data. 

Step 2: Construct an RDF representation of the relational data. At this step we 
are dealing with the actual table content - the cell values. We consider every 
row as an instance of H, where every cell is represented as an instance of a class 
corresponding to its column. In addition, we instantiate all the virtual relations 
defined on the previous step. 

This step provides us with instances representing table data and once hidden 
but now explicit relationships between cells of a row. At this point we have the 
advantage of being able to use general-purpose RDF /S repositories and querying 
engines but we still cannot employ the vocabulary defined in DO to access the 
relational data. 

Step 3: Ask the user to link RS with the domain ontology. The user links concepts 
(classes and properties) from the relational schema to corresponding concepts in 
the domain ontology by identifying rdf s : subClassOf and rdf s : subPropertyOf 
relationships between corresponding classes and properties. A set of all such links 
constitutes an RDMap (Relational- RDFS Map). The RDMap directly links the 
relational schema to ontological definitions. 

Having obtained the RDF/S serializations of the relational data and the 
RDMap, an RDFS reasoner will be able to deduct all necessary entailments to 
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perform the actual linking of the relational schema and data with the concepts 
defined in DO. We will illustrate this with an example in the next subsection. 
The RDFS reasoner is required to merge the separate RDF/S documents and to 
generate the entailments. 

To test the proposed technique and to provide basic support to the user we 
have developed FDR2#Kit 1 - a web-based toolkit consisting of a few utilities: 
FDR2# Generator takes a tab-delimited text file with tabular data and auto- 
matically generates RDF/S documents for the relational schema and relational 
d&t&\FDR2#Mapper assists the user with linking the relational schema to DO; 
FDR2# Tester allows to run simple queries over the resulting combination of 
schema, data, RDMap and DO. 

3 Discussion and Related Work 

Over-generated virtual relations pose a serious performance problem. For exam- 
ple, a 10-column table will result in a RS with 90 virtual relations and quite 
significant portion of them may be redundant. Such a RS does not require a 
lot of resources to handle but a corresponding RDF-serialization of the table 
content will be polluted with irrelevant data. This problem can be handled by 
introducing a separate step between automatic generation of the RS and serial- 
ization of the table content. This stage is needed to enable the user to remove 
the redundant virtual relations from the original RS. Another option would be 
to swap steps 2 and 3 and to exploit a created RDMap to (semi)-automatically 
remove virtual relations not linked with DO. The modified RS will determine 
the final structure of the table content serialization preventing from polluting it 
with irrelevant data. 

The relational schema facilitates analysis of the relational data on an ab- 
stract, intensional level. A possible practical applications of this is that an 
RDFS-based information system can keep track of known relational schemata 
and corresponding linking maps. This allows automating the whole process of 
handling complex input data. Since the RS is constructed automatically, it is 
quite likely that once created, the RDMap can be reused by many users who 
even do not know anything about the details of linking procedure and still able 
to take advantage of RDFS inference. 

In [7] the authors describe a “naive” approach for mapping RDBMS schemata 
onto RDF (although we would rather call it RDBMS data mapping onto RDF). 
Our work takes it to the next level where we are focusing on linking relational 
and RDFS schemata. RDF serialization of actual data is quite straightforward 
and can be done in different ways according to application specific restrictions. 

4 Conclusions 

In this paper we have introduced FDR2 a technique that enables us to link 
relational and RDF/S data models. According to FDR2 a relational schema 

1 http:/ /www. cs.vu.nl/Anaksym/fRools 
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is automatically created to explicate the structure and internal relationships 
between elements of a relational collection of data. Explication of virtual re- 
lations allows the user to construct a relational schema specific RDMap by 
defining rdf s : [subclass I Property] Of relationships between concepts from the 
relational schema and a domain ontology. The actual relational data are au- 
tomatically expressed in RDF according to the generated relational schema. 
Run-time integration is achieved by applying an RDFS reasoner to merge the 
above-mentioned components into a single RDFS model and to deduct neces- 
sary entailments. A resulting run-time model allows to access the relational data 
with queries termed according to the domain ontology. FDR2 is purely RDF/S- 
based and does not require any additional software components except an RDFS 
reasoner. 
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Abstract. This paper describes the new e-learning tool CHESt that allows stu- 
dents to search in a knowledge base for short (teaching) multimedia clips by 
using a semantic search engine. We explain the different steps to automatically 
describe the meaning of the clips with RDF ( Resource Description Frame- 
work). The concept is based on graph theory and retrieval algorithms. Finally, 
we present rules how a human question can be transformed into a RDF query. 
Thus, the knowledge base and the query have the same format and can be com- 
pared. 



1 Our Multimedia E-learning Tool 

CHESt ( Computer History Expert System ) is the prototype of a new e-learning tool, 
see [7] for details. It focuses on three key features: the information is in a multimedia 
form, the content is split into small clips and a semantic search mechanism for infor- 
mation retrieval. We used Tele-TASK [ 1 ] [2] to record the lessons in order to create 
one well-structured multimedia stream. The result is a large number of RealMedia 
files that can be played with any compatible software, for example the free RealOne 
Player [5]. 

Essential in our concept is the length of the stored items in the knowledge base; the 
duration of the multimedia sequences. The younger the user, the shorter the time 
during which he/she will concentrate on the information displayed on the screen. 
Furthermore, it is easier to find the appropriate information inside a small piece of 
data than for example in an online lesson that lasts 90 minutes. Thus, we divided all 
our multimedia data into small clips. The duration of each clip varies from several 
seconds to 3 or 4 minutes. Each clip documents one subject or a part of a subject. 
Together, all the clips of the knowledge base cover one large topic; in our prototype 
we focus on computer history. We produced more than 300 clips about most impor- 
tant events in computer history. CHESt exists as standalone application (knowledge 
base and application software on one CD-ROM) and as online application. The later 
uses a streaming server to transmit the clips to the user's browser. 

In this paper we present a retrieval mechanism where the user can enter a complete 
question. The tool "understands" that question and gives a small list of pertinent clips 
as answer, or better even just one clip. 
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2 Describing the Meaning of the Clips 

However, before the tool can even try to understand the user's question, it has to 
"know" what data are stored in the knowledge base. Therefore, we have to add meta- 
data to each clip to describe its meaning. For this purpose we use the Resource De- 
scription Framework (RDF) [10]. In principle, this is done once, at the moment when 
the clip is added to the knowledge base. However, the computer can take on a part of 
this task. 

We divided the CHESt knowledge base logically into two classes: clips that de- 
scribe inventions (things) and clips that describe inventors (persons). Assertion: an 
invention was invented by one or more inventors. An invention and an inventor can 
be a resource (in our case: a clip) or a literal (just a textual information). Every re- 
source is described with properties. An inventor has three properties ( predicates ): his 
name (vcard : FN), the year of his birth (chest : year_birth) and the year of his 
death (chest :year_death); if still alive, this property is left blank. As you see, 
we used the W3C recommendation vCard namespace property full name (FN) [9], 
The class invention is divided into a number of subclasses to better organize the 
different resources (for example: Hardware, Software...). We used the Dublin Core 
(dc) namespace [3] to describe an invention with the following properties (predi- 
cates): its description (dc: title), its date of first appearance (dc : date) and its 
inventor (dc : creator). The complete CHESt RDF schema can be found at [6], 

The next step is to search inside every clip for metadata. We applied an approved 
approach from the field of computer linguistics: create a dictionary of synonyms for 
every CHESt RDF element [4] [8]; in one column one will find the RDF elements and 
in the other column there is a list of natural language synonyms. For example, if we 
are scanning for dc : creator, we are searching for words like creator, builder, 
constructor, inventor, etc. The slides used to create the Tele-TASK clips are con- 
verted into pure text files. Then the stemming process can begin. All non-words and 
words with just one letter were eliminated from the generated text files because they 
have no semantic influence. All words are converted into lowercase and separation 
characters {,-.?!() + */ &@J are replaced by a space. Then, a tree is built from 
those words, where every node represents one letter (see figure 1). This technique 
also allows to eliminate all double words. Each node contains the number of words 
that end with that particular letter. 

The dictionary of synonyms is built from that tree. The idea is to regroup words 
with a similar spelling and thus with the same meaning (for example: build, built, 
builds). It is impossible to detect automatically all synonyms, because there are words 
that have a similar spelling, but not the same meaning. The aim of the stemming pro- 
cess is to limit human intervention by proposing clusters of generated synonyms. We 
got acceptable results with three simple rules. Two words are synonyms only if all 
three rules match. 

• Firstly, the common part (trunk) of two words must have a length of at least 4 
letters, for example: trunk(begin, beginning ) = {begin} >4. 

• Secondly, the remaining and not common part (tail) must not be longer than 5 
letters, for example: tail(begin, beginning) = {ning}< 5. 
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• Thirdly, a different letter is only accepted if the common part has at least 3 letters, 
for example: trunk(begin, began) = [beg} >3. 

Finally, RDF elements were affected to the concerned clusters, for example the clus- 
ter containing the words { begin, begins, beginning, start, starting } becomes synonym 
for dc : date and the words [inventor, builder, constructor, inventors} are affected 
to dc : creator. The final clustered dictionary is stored for later use (see section 3). 
The final step consists in scanning through the clips and searching for synonyms for 
the RDF elements. The result is a RDF/XML serialization for each clip. 




Detected synonyms: 



beg 

begin 

begins 

beginning 

began 

behalf 

behind 

believe 



Fig. 1 . Example of a generated tree of words. The number in brackets indicates the number of 
occurrences of the word. If the number is zero, then this node is no final letter. In this gener- 
ated example, no wrong synonym is found. But one synonym was not clustered: [began] 
should be placed in the cluster [begin, begins, beginning] . 



3 Understanding the User 

To perform a semantic search, the question entered by the user must be transformed 
into RDF, in order to have the same structure for the question and for the database. 
The backbone of our semantic search is an inference engine which transforms a nor- 
mal sentence (the user’s question) into a well-formulated RDF query. For example: 
“Who invented the very first calculator” should become: 

SELECT <?x> WHERE <chest : Computer> ; <dc : creator> ; <?x> 

We will not describe details about representing RDF data in a database or how to 
launch a RDF query; see for example [11]. We will focus on the parsing of the sen- 
tence and the construction of the RDF query. 
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Table 1. Illustration of the basic rule for transforming a user's question into a RDF query. 



Question 


Subject 


Predicate 


Object 


Who built the first calculator? 


chest : Computer dc: creator 


?x 




(i calculator ) 


(built) 




What does Zuse invent? 


?x 


dc : creator 


chest : Person 






(invent) 


(Zuse) 


Table 2. Illustration of several exceptions for transforming a user's question into a query. 


Question 


Subject 


Predicate 


Object 


When was Aiken born? 


chest : Person 


chest :year_birth 


?X 




(Aiken) 


(born) 




What was the year Aiken 


chest : Person 


chest :year_death 


?X 


died? 


(Aiken) 


(died) 




What does ARP A mean and 


chest : Firm 


dc : creator 


?x 


who founded it? 


(ARPA) 


( founded ) 






chest : Firm 


dc : title 


?x 




(ARPA) 


(mean) 




Who built the ENIAC and the 


chest : Computer 


dc : creator 


?x 


EDVAC? 


(ENIAC. EDVAC) 


(built) 




When did Zuse build his Z3? 


chest : Computer 


dc : creator 


?x 




(Z3) 


(build) 






?x 


dc : creator 


chest : Person 






(build) 


(Zuse) 


What is Linux? 


chest : OS 








(Linux) 







The transformation of a common formulated sentence into RDF can be summed up 
by saying that the system has to replace all semantically important words by the RDF 
corresponding elements and to throw unimportant words away. For the question 
“Who invented the very first calculator?” the following words were replaced: { in- 
vented ] — > dc : creator, { calculator }— > chest : Computer. All other words will 
not be considered. The missing part becomes the subject of the query. See table 1 for 
some general examples. But there are a lot of imaginable exceptions, for example: 

• The predicate is not dc : creator (see table 2, lines 1+2). In that case, we are 
not in the basic assertion: "An invention was invented by an inventor", thus the 
general rule cannot be applied. It is a fact that the missing part must be the object. 
It is also a fact that the user is not searching for a person or an invention. There 
are several possible predicates depending on the class-membership of the subject: 
{dc:date and dc: title} if the subject is an invention or 
{chest :year_birth, chest : year_death and vcard:FN} if the subject 
is an inventor. The parser must choose the right predicate by analyzing the other 
found synonym(s), for example: words like "born" or "died" indicate a date. 

• There is more than one predicate in the sentence. If the predicates are not concur- 
rent then there will only be one query. If there are concurrent predicates (see table 
2, line 3) then there will be as many queries as there are different predicates. 
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• There is more than one subject or object in the sentence. In analogy to the above 
exception, if the subjects or objects are not concurrent (see table 2, line 4). there 
will only be one query. If there are concurrent subjects or objects then there will 
be as many queries as there are different subjects or objects. 

• There is no missing part. The question contains a predicate, a subject and an ob- 
ject (see table 2, line 5). This is the most complicated exception to handle. The 
system must find the best matching clip by associating the different queries. 

• There are less than two known parts. In that case, the system lists all resources 
matching the keywords for the given class (see table 2, line 6). 

4 Outlook 

The prototype CHESt is tested with a simple keyword search in some selected schools 
in the summer term of the year 2004. Meanwhile, we are working on the improve- 
ment and development of the semantic search engine described in section 3. A proto- 
type is to be tested in a larger pilot project in several schools and universities for the 
coming winter term (interested schools can contact us). The experience and empiric 
data that will be collected with the educational tool CHESt should then be the base of 
further research for a more general semantic search engine. One could imagine devel- 
oping a generalized interface to access the knowledge base that contains clips of dif- 
ferent topics: geography, French vocabulary, irregular English verbs, explanation of 
HTML tags, biography about famous actors, etc. 
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Abstract. There is a need to integrate XML schemas, i.e. , schemas writ- 
ten in XML Schema, into UML-based software development processes. 
Not only the production of XML schemas out of UML models is re- 
quired, but even more the integration of given XML schemas as input 
into the development process. In the model driven architecture, a two 
step integration is assumed, comprising a platform specific model and a 
platform independent model. Several approaches already exist addressing 
the problem of automatically creating a platform specific model for XML 
schemas. This paper contributes a comparison of these approaches, based 
on a comprehensive set of transformation patterns supporting creation 
of a platform specific UML model that is as concise and semantically 
expressive as possible without loosing XML Schema information. 



1 Introduction 

UML is being used as de-facto standard for software development, including web 
applications that exchange XML documents. Therefore a need arises to integrate 
XML schemas, i.e., schemas written in XML Schema, into UML-based software 
development processes. Not only the production of XML schemas out of UML 
models is required, but even more the integration of XML schemas as input into 
the development process, because standard data structures and document types 
are part of the requirements. 

In the model driven architecture [9], a two step integration is assumed, com- 
prising a platform specific model which abstracts from implementation language 
details, and a platform independent model which abstracts from technology de- 
tails. For the platform independent model, plain UML is applied, whereas for 
the platform specific model, UML tailored to the target technology is employed. 
An evaluation of existing UML profiles for XML Schema as possible target tech- 
nology is the main contribution of this paper. 

A UML profile for XML Schema must fulfill several requirements. In par- 
ticular, we are looking for a semantically equivalent representation of an XML 
schema in UML supporting a bijective mapping between both representations. 
A solution to this problem has to address the whole range of XML Schema 
concepts, such that any XML schema can be expressed in UML. Another re- 
quirement is to support round-trip engineering, i.e., transformation from XML 
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Schema to UML and back again without loss of schema information. Further- 
more, a solution should maximize understandability of semantic concepts by 
users knowledgeable of UML but not XML Schema. Semantic equivalence is 
even more important when the UML models are to be used for application code 
generation, as it will happen in a model-driven development process. 

This paper compares five main approaches for representing XML Schema in 
UML. The features of the approaches are compared based on a comprehensive 
set of transformation patterns fulfilling the above identified requirements. The 
patterns have been extracted from a previous effort to define a UML profile ([1]). 
In the next section, an overview of existing approaches is given, followed by the 
comparison results and a description of the transformation patterns in the final 
section. 



2 Overview of Approaches 

Existing work on representing XML Schema in UML has emerged from ap- 
proaches to platform specific modeling in UML and transforming these models 
to XML Schema, with the recognized need for UML extensions to specify XML 
Schema peculiarities. [2] is the first approach of this kind to modeling XML 
schemas using UML. Although based on a predecessor to XML Schema, it intro- 
duces UML extensions addressing modelling of elements and attributes, model 
groups, and enumerations that can also be found in following approaches. 

The approach by Carlson ([3]) describes an approach based on XMI rules 
for transforming UML to XML Schema. [3] also defines a UML profile which 
addresses most XML Schema concepts, except of simple content complex types, 
global elements and attributes, and identity constraints. Regarding semantic 
equivalence, the profile has some weaknesses in its representation of model 
groups, i.e., sequence, choice, and all. Based on the profile defined in [3], a 
two-way transformation between XML Schema and UML has been implemented 
in the commercially available tool “hypermodel” 1 . 

In [10], Provost has addressed some of the weaknesses of [3], addressing rep- 
resentation of enumerations and other restriction constraints, and of list and 
union type constructors, although the latter doesn’t conform to UML. 

Eckstein’s approach ([5], in german, based on [4]) also defines a profile similar 
to that in [3], with some enhancements regarding simple types and notations. 

Goodchild et al (in [8]) point out the importance of separating the concep- 
tual schema, i.e., the platform independent model, from the logical schema, i.e., 
the platform specific model, a separation that is not considered in the other ap- 
proaches. In [8], the logical schema is a direct, one-to-one representation of the 
XML schema in terms of a UML profile. The profile 2 covers almost all concepts 
of XML Schema, but several of its representations are not UML conform. 

1 http : / / xmlmodeling . com/hyperModel/ 

2 A complete description can be found at 

http : / /titanium. dstc . edu.au/papers/xml-schema-profile .pdf 
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Our approach (in [1]) follows [8] in that it also aim at a one-to-one represen- 
tation of XML schemas in an UML profile. The approach builds on the existing 
UML profiles for XML Schema, with some improvements and extensions. 

Related work on mapping conceptual models expressed in UML or EER 
to XML Schema or DTD, has also identified various options for transforming 
conceptual-level concepts to XML Schema concepts [3,4,6,7,10]. Most of these 
transformations are, however, not unambiguously applicable in the reverse di- 
rection and would thus only be useful in an interactive transformation process, 
requiring a user’s knowledge of the XML schema to be transformed to UML. 
Therefore, these approaches are not evaluated in this paper, although some of 
their results have influenced the design of the transformation patterns. 

3 Comparison 

A comparison of the features of each approach is provided in Table 1, organized 
along the various transformation patterns as described below. As can be seen, 
most of the approaches fail to fulfill the requirement of supporting all XML 
Schema concepts. Furthermore, some approaches represent XML Schema con- 
cepts in UML in a way supporting syntactic transformation but failing to provide 
semantic equivalence. [1] provides a solution satisfying the requirements, with 
the main improvements being solutions to represent model groups (MG) as well 
as global elements (EG, EW) and global attributes (AG, AW) in a way more com- 
pliant to UML semantics, to represent identity constraints (KY), and to represent 
simple types in a more concise, UML like way (ST3-4). For technical details on 
the differences of the approaches it is referred to [1], 



Table 1 . Comparison of UML profiles by transformation patterns 
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3.1 Transformation Patterns 

Three design goals have guided the design of transformation patterns. First, it 
must be possible to represent any XML schema in UML, i.e., there must be 
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a representation for each relevant XML Schema concept, in order to facilitate 
round-trip engineering without loss of schema information. Second, a representa- 
tion of an XML schema has to be such that if the profile specific stereotypes are 
omitted, the result should - to the extent possible - convey the same meaning, 
in order to facilitate understanding by non-XML Schema experts and to support 
interoperability with tools not aware of the profile. This goal is also in line with 
the capability of UML stereotypes, which can only extend but not modify the 
semantics of UML concepts. Finally, the number of UML constructs necessary 
to represent a certain XML schema should be minimal , to improve readability. 
This goal can be achieved in some situations where UML concepts are more 
expressive than XML Schema concepts, allowing to represent certain patterns of 
XML Schema concepts using only one UML concept. 

The transformation patterns are organized along the major XML Schema 
concepts, i.e., schema, complex types, simple types, elements, attributes, model 
groups (i.e., complex content), identity constraints, group definitions, annota- 
tions, and notations. A more detailed description of the transformation patterns 
can be found in [1]. 

SC Represent every schema document as a stereotyped package. 

CT1 Represent every global complex type as a stereotyped class. 

CT2 Represent every local complex type as a stereotyped class nested into its 
containing class. 

ST1 Represent every simple type that includes an enumeration constraint as a 
stereotyped enumeration. 

ST2 Represent every simple type that does not include an enumeration con- 
straint as a stereotyped primitive datatype. 

ST3 Simplification of ST1/2: Merge the representation of a local simple type 
that is the type of an UML attribute with that attribute. 

ST4 Simplification of ST1/2: Merge the representation of a list or union type 
defined local to a restriction type with the representation of the latter. 
ELI Represent a local element as a stereotyped association role of an association 
connecting the element’s containing model group with the element’s type. 
EL2 Represent a local element as a stereotyped attribute of the class represent- 
ing the containing model group if the element’s type is a simple type. 

EG Represent every global element like a local element declaration with an 
additional stereotyped class. Represent its usage by generalizations to that 
class. 

EW Represent every element wildcard as a multiple classification constraint, 
indicating that occurrences can be instances of global elements as well. 

AL Represent every local attribute as a stereotyped attribute of the class rep- 
resenting the containing complex type or group. 

AG Represent every global attribute like a local attribute with an additional 
stereotyped class. Represent its usage by generalizations to that class. 

AW Represent every attribute wildcard as a multiple classification constraint, 
indicating that occurrences can be instances of global attributes as well. 
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MG1 Represent a model group as a stereotyped class where the stereotype de- 
pends on the group’s compositor. Represent its usage by a composition. 

MG2 Represent the grammar expressed by a model group tree as a constraint, 
using a textual notation which covers hierarchical structuring and ordering. 

KYI Represent a key constraint as a constraint attached to the class containing 
the representation of the element that is the key’s scope. 

KY2 Represent a key constraint whose selector does not contain union and 
wildcard steps as a constraint attached to the class selected by the selector. 

GA Represent every attribute group as an abstract stereotyped class with the 
attributes represented by AL. Represent its usage by generalizations. 

GE1 Represent an element group as a stereotyped class with the elements repre- 
sented by ELI and/or EL2. Represent its model group and usage by MG1. 

GE2 Represent an element group as an abstract stereotyped class with the ele- 
ments represented by ELI and/or EL2. Represent its model group by MG2 
and its usage by generalizations. 

AN Represent every annotation as a set of stereotyped comments. 

NO Represent every notation declaration as a stereotyped literal in a stereo- 
typed enumeration. 
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Abstract. Travelling upon the Web is difficult for visually impaired users since 
the Web pages are designed for visual interaction [6]. Visually impaired users 
usually use screen readers 1 to access the Web in audio. However, unlike sighted 
users, screen readers cannot see the implicit structural and navigational knowledge 
encoded within the visual presentation of Web pages. Therefore, in a visually im- 
paired user’s environment, objects that support travel are missing or inaccessible. 
Our approach to remedy this is to annotate pages with an ontology, the Travel On- 
tology, that aims to encapsulate rich structural and navigational knowledge about 
these objects. We use Semantic Web technologies to make such knowledge explicit 
and computationally accessible. Our semi-automated tool, Dante identifies travel 
objects on Web pages, annotates them appropriately with the Travel Ontology and 
uses this to transform the pages to enhance the travel support. Thus Dante uses 
the Travel Ontology to enhance the travel experience of visually impaired users. 
This paper introduces the Travel Ontology, the annotation pipeline used in the 
annotation part of Dante and some transformation scenarios to illustrate how the 
annotations are used to guide the transformation of Web pages. 



1 Introduction 

This paper introduces a semi-automated tool, Dante , for the support of travel and mo- 
bility for visually impaired Web users. The paper first presents an ontology, the Travel 
Ontology, and the annotation pipeline facilitated within Dante, and then discusses how 
the Travel Ontology is used to transform pages to enhance the travel experience of 
visually impaired Web users. 

The visual navigational objects that support easy movement around Web pages, 
or mobility, are not appropriate and accessible to visually impaired Web users. These 
objects are crucial to confident, easy and accurate navigation, which we call travel [6]. 
In order to support mobility, these objects and their roles need to be identified, explicitly 
specified and presented in a way to fulfil their intended roles. The idea behind Dante is to 
analyse Web pages to extract such objects and annotate them with terms from the Travel 

1 Screen readers are special applications that vocalise the onscreen data. Pages are typically read 
from the top left to the bottom right, line by line, one word at a time [6]. 
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